Improve file type detection of NetCDF and HDF5

[pdurbin]

According to Wikipedia NetCDF and HDF5 have magic numbers that should let us detect these file types more easily and reliably than guessing based on file extensions.

NetCDF magic number

CDF\001
\211HDF\r\n\032\n

HDF5 magic number

\211HDF\r\n\032\n

I brought this up at standup today and here are some notes from the discussion:

• We should see if JHOVE can detect them.
• Normally, detecting file types by seeking into files is part of detecting tabular files, specifically.
• Given that NetCDF can be big, this might be a case where switching to a ranged request to find the signature might be important.

We should add some NetCDF and HDF5 files to https://github.com/IQSS/dataverse-sample-data to test with, at some point.

Related:

• HDF5 files show type "unknown" dataverse.harvard.edu#23

[mreekie]

Updated Project status's