geotiff: auto-promote float16 reads to float32 (#1941)#1944
Merged
Conversation
The writer auto-promoted float16 inputs to float32 before encoding, but the read-side dtype map in _dtypes.tiff_dtype_to_numpy had no entry for (16, SAMPLE_FORMAT_FLOAT), so any externally produced GeoTIFF with half-precision samples failed at parse time with "Unsupported BitsPerSample=16, SampleFormat=3". rasterio / GDAL handle these files; xrspatial did not. Fix: - tiff_dtype_to_numpy(16, 3) returns np.float32 (matches the writer's promotion contract). - A new tiff_storage_dtype(bps, sf) returns the raw on-disk dtype (np.float16 for this key, identical to tiff_dtype_to_numpy for every other key). _decode_strip_or_tile uses the storage dtype to view the 2-byte samples then casts to float32 once. - GPU paths fall back to CPU decode for float16 (bps != itemsize * 8) because the GDS/tile-assembly kernels assume bps matches the user-visible dtype; the existing stripped-layout fallback handles the fallback dispatch. Regression test in test_float16_read_1941.py: dtype-map keys, eager and dask round-trip of an external float16 file, predictor=3 + float16, plus regression guards for float32/float64/uint16. Existing test_edge_cases.test_float16_not_supported is updated to assert the new auto-promotion behaviour.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1941.
Summary
tiff_dtype_to_numpy(16, SAMPLE_FORMAT_FLOAT)previously raised; an externally-produced GeoTIFF withBitsPerSample=16+SampleFormat=3could not be read.tiff_storage_dtypekeeps the raw 2-byte view so the byte view in_decode_strip_or_tilestays correct.Test plan
tiff_dtype_to_numpy(16, 3) == float32,tiff_storage_dtype(16, 3) == float16, identity on non-promoted keys.open_geotiffandread_geotiff_daskround-trip a tifffile-written float16 file to float32.test_float16_not_supportedis updated to assert the new behaviour.pytest xrspatial/geotiff/tests/non-GPU/HTTP: 2382 passed, 7 skipped.