Add new guide on netCDF file format#24
Conversation
hcronk
left a comment
hcronk
left a comment
There was a problem hiding this comment.
Thanks for getting this going, Veronica!
|
|
||
|
|
||
| ## Purpose for this guideline | ||
| NetCDF is a file format commonly used at LASP... |
There was a problem hiding this comment.
It is the "highly preferred" format for NASA Earth Observing System Data and Information System data products, per their Data Product Development Guide for Data Producers. This affects all NASA Earth Science missions
| * long_name | ||
| * units | ||
| * Conventions | ||
| * [Climate and Forecast (CF)](https://cfconventions.org/Data/cf-conventions/cf-conventions-1.8/cf-conventions.html) |
There was a problem hiding this comment.
It would be useful to include information for compliance checkers, especially since this is the convention being pushed by the NASA Earth Science Data Systems and that changes what's required vs. useful attributes
| #### Other useful variable attributes | ||
| * missing_value | ||
| * prefer over _FillValue | ||
| * NaN is a good option |
There was a problem hiding this comment.
I disagree -- NaNs in files are handled differently in every language and I feel like it's better to pick a value for official data products that many users will be using
There was a problem hiding this comment.
I've updated this section. A quick google search indicates that missing_value is considered deprecated. I also added a note on NaNs based on your input. This is helpful insight, thanks for sharing!
| ## Useful Links | ||
| * [NetCDF User's Guide](https://docs.unidata.ucar.edu/nug/current/) | ||
| * [NetCDF ToolsUI](https://docs.unidata.ucar.edu/netcdf-java/current/userguide/toolsui_ref.html) | ||
|
|
There was a problem hiding this comment.
There was a problem hiding this comment.
I'll add this url to the list of links, thanks!
| * Requires software libraries to read and write C, Fortran, Java, python, IDL, ... | ||
| * Internal compression, don't bother to compress NetCDF files externally | ||
| * HTTP byte range requests | ||
| * Parallel IO |
There was a problem hiding this comment.
This could be a rabbit hole, depending on how much you want to get into things like Dask
bourque
left a comment
bourque
left a comment
There was a problem hiding this comment.
This is a great start! And thanks for detailing what else still needs to be done to this document (feel free to open a new issue for that).
Could you also add a path to this file in the index.rst file so that it appears in the RtD build?
|
I made some minor updates and added the guide to index.rst. I'll create a ticket for content still needed |
* New guide on netCDF file format. Additional content needed * Add hyperlinks * Minor edits to provide some clarity. Still needs an introduction for each section * Add guide to read the docs. Incorporate feedback from PR
3 participants
Created a guide about the netCDF file format pulling content written by Doug Lindholm. Confluence page
The content is mostly in the structure of bullet points. Some additional text is needed to provide context and explain how this file format is being used at LASP. I added a warning at the top of the guide noting that it's incomplete.
Details still needed:
Credit is given to Doug at the bottom of the file, but we should figure out how best to do this since the structure is different from his original page