Skip to content

BLD: cfchecker update travis#966

Closed
cpelley wants to merge 1 commit intoSciTools:masterfrom
cpelley:rm_custom_cf_checker
Closed

BLD: cfchecker update travis#966
cpelley wants to merge 1 commit intoSciTools:masterfrom
cpelley:rm_custom_cf_checker

Conversation

@cpelley
Copy link

@cpelley cpelley commented Jan 21, 2014

Currently travis is failing as it is unable to return the tables from http://cf-pcmdi.llnl.gov
This PR updates the cfchecker, thereby using the cf standard names table distributed with the cfchecker itself, rather than relying on http://cf-pcmdi.llnl.gov

@cpelley
Copy link
Author

cpelley commented Jan 21, 2014

I think this is ready

@bblay
Copy link
Contributor

bblay commented Jan 21, 2014

@bjlittle, @marqh, please give this a quick viddy before I merge.

@ghost ghost assigned bblay Jan 21, 2014
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit concerned this isn't doing quite what it says on the tin.
I understand that we need cfchecker because test_pp_cf uses it.
But as far as I can see, it does not use its own copy of cf-standard-names.xml by default
-- instead, it will fetch it from STANDARDNAME = 'http://cf-pcmdi.llnl.gov/documents/cf-standard-names/standard-name-table/current/cf-standard-name-table.xml' (from cfchecker code)
In which case, it seems odd that this works at all, as it is ??

Meanwhile, what we used to do here was

  1. download a specific table and push it into lib/iris/etc
  2. create a custom cfchecker that refers specifically to that

I don't think we should be ditching that approach entirely here, so can we at least keep the old version commented out ?

I'm puzzled as to how the cfchecker test can be working now, if cfchecker is itself downloading from pcmdi.
As it happens, though, we do already have a copy of the version 23 'cf-standard-names.xml' burnt into the Iris repo at lib/etc
So I think it would be better to -

  1. comment out the xml download for now
  2. revert to the previous method of providing a 'custom' cfchecker, where the source is loaded from pypi (though you can up the version number if it looks good -- it seems surprising there is no 'latest stable' link, but I think you're right + there is not)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After some offline chatting, I'm wondering why travis doesn't use the same version of the tables as the rest of us: https://github.com/SciTools/iris/tree/master/etc

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it seems odd that this works at all, as it is ??

From the Travis log:

/home/travis/build/SciTools/iris/lib/iris/tests/test_pp_cf.py:119: UserWarning: CF-netCDF "cfchecker" application not available. Skipping CF-netCDF compliance checking.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In other words, the tests are looking for an executable named cfchecker, which is our own creation. It is a wrapper to cfchecks which sets the location of the lookup tables and UDUNITS. Having removed the creation of this script in this PR all the relevant tests are just skipping it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @pp-mo for offline chatting, and to @rhattersley.
If we can dig out a copy of area-type-table.xml, I suggest we add it to the Iris codebase,
and stop using this cfchecker script, using instead a call to cfchecks with pointers to Iris' copies of the files.

Advantages:

  • no deps on breakable web resource
  • anyone can run these tests, not just ukmo & travis

Any objections?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stop using this cfchecker script, using instead a call to cfchecks with pointers to Iris' copies of the files

That's a separate issue from the one of the table downloads. I'd really like to keep it so.

@rhattersley
Copy link
Member

After some offline chatting, I'm wondering why travis doesn't use the same version of the tables as the rest of us: https://github.com/SciTools/iris/tree/master/etc

Quite. Although we'd also need to include the area type table if we're to avoid being hamstrung by the flaky nature of the pcmdi.llnl.gov servers.

@rhattersley
Copy link
Member

we'd also need to include the area type table if we're to avoid being hamstrung by the flaky nature of the pcmdi.llnl.gov servers.

See #968 for an attempt at this.

@bblay
Copy link
Contributor

bblay commented Jan 21, 2014

See #968 for an attempt at this.

Ah! You know what they say, "Great minds think alike". (Conincidentally, so do ours.)

@bjlittle
Copy link
Member

@cpelley I've just had a quick whizz through the cfchecker code and @pp-mo is right in that no tables are bundled with the checker package.

It simply downloads the standard-name and area-type tables from:

  • http://cf-pcmdi.llnl.gov/documents/cf-standard-names/standard-name-table/current/cf-standard-name-table.xml and
  • http://cf-pcmdi.llnl.gov/documents/cf-standard-names/area-type-table/current/area-type-table.xml

using an xml.sax parser, and it also points to the default location for udunits2 .xml file if not provided.

So there's still no avoiding the flaky nature of the pcmdi.llnl.gov servers here ...

@cpelley
Copy link
Author

cpelley commented Jan 21, 2014

@cpelley I've just had a quick whizz through the cfchecker code and @pp-mo is right in that no tables are bundled with the checker package.

That's annoying, I assumed it was ok because travis tests passed. I'm assuming it momentarily became available.

@cpelley cpelley closed this Jan 21, 2014
@cpelley cpelley deleted the rm_custom_cf_checker branch January 21, 2014 15:18
@bblay
Copy link
Contributor

bblay commented Jan 21, 2014

I'm assuming it momentarily became available.

No, see @rhattersley's comment above, it skipped the test

@rhattersley
Copy link
Member

I'm assuming it momentarily became available.

Please read my earlier comment. The tests "passed" (or rather were skipped) because you deleted the cfchecker script.

@rhattersley
Copy link
Member

#968 has worked ... I propose we merge that and leave this one closed.

@cpelley
Copy link
Author

cpelley commented Jan 22, 2014

OK

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants