Reason or Problem
The geotiff module has ~25k LOC of tests against ~17k LOC of parser/writer code, but every test in xrspatial/geotiff/tests/ is example-based: handwritten inputs built with make_minimal_tiff from conftest.py. Zero property-based or fuzz tests.
Recent commits show a pattern of bugs that property-based testing catches cheaply:
babb72e _coords_to_transform broke for 3D (y, x, band) DataArrays
03d7380 georef not inherited when reading overview IFDs
c294578 / dd907b8 nodata sentinel poisoned cubic / COG overviews
9a5f55e read_vrt leaked integer source nodata into a float-dtype VRT
f0a09c0 write_geotiff_gpu(file_like, cog=True) silently accepted an unsupported combination
595ece8 signature/docstring drift between to_geotiff, write_vrt, write_geotiff_gpu
dbb5ceb user-defined CRS WKT not promoted to attrs on read
cdf7d43 LERC / JPEG 2000 decompression output not capped (block bomb)
These cluster into a few classes:
- Kwarg silently dropped or routed to the wrong code path
- Per-band / per-chunk metadata leaking (nodata, dtype)
- Georef / attrs not inherited when traversing IFD chains
- Round-trip equality broken for an unusual but valid combination of (dtype, compression, tiled, predictor, nodata)
Hand-written tests catch one variant at a time. Property tests cover the cross-product cheaply.
Proposal
Add Hypothesis-based property tests for the geotiff module.
Design:
A new test file xrspatial/geotiff/tests/test_fuzz_hypothesis_<issue>.py with three groups:
-
Round-trip property. Strategies generate (width, height, dtype, compression, tiled, predictor, nodata), build a DataArray, write with to_geotiff, read with open_geotiff, assert array equality and attrs preservation. Bounded ranges (dims 1 to 32, a short dtype list, common codecs) so CI stays fast.
-
IFD layout permutations. Use make_minimal_tiff with strategies over tag order, tile vs strip, big/little endian, with/without geo tags. Assert open_geotiff either returns a valid array or raises a typed exception from the geotiff module. Never a bare IndexError / struct.error / UnicodeDecodeError.
-
Byte-level mutation fuzz. Take a valid TIFF from make_minimal_tiff, flip one byte at a Hypothesis-chosen offset, assert the reader either parses consistently or raises a typed exception. Never silently returns wrong data, never segfaults.
Each test uses @settings(max_examples=50, deadline=None) to bound CI time.
Hypothesis is not currently a test dependency. The implementation will guard the import with pytest.importorskip('hypothesis') so the suite still runs when hypothesis is missing. A follow-up PR can add it to setup.cfg [tests] once the harness has earned its keep.
Usage:
pip install hypothesis
pytest xrspatial/geotiff/tests/test_fuzz_hypothesis_*.py -v
Locally, --hypothesis-show-statistics shows shrinking behaviour on failure.
Value:
- Catches the next instance of every bug class above before a user files it
- Documents the parser's exception contract (what is "invalid input" vs "our bug")
- Cheap maintenance: strategies are short, seeds make failures reproducible
Stakeholders and Impacts
Geotiff module maintainers. No runtime behaviour changes, no new public API. Impact limited to the test suite.
Drawbacks
- Hypothesis adds ~1s to test collection time when installed
- Bugs found by the fuzzer need triage (real bug vs strategy too broad)
- Property failures are harder to read than example failures until the counterexample shrinks
Alternatives
- Keep adding example-based tests one bug at a time (status quo)
- AFL or
afl-tiff: heavier infrastructure, harder to run in CI
- Run hypothesis in a nightly CI job only: a possible follow-up if collection time hurts
Unresolved Questions
- Add
hypothesis to setup.cfg [tests] now or keep it opt-in? Defaulting to opt-in for this PR.
- Seed a real-file corpus (GDAL COG, Sentinel-2-style)? Out of scope here; open a follow-up if synthetic strategies leave gaps.
Additional Notes or Context
Audit numbers:
- Parser/writer LOC: 17,314 (
xrspatial/geotiff/_*.py + __init__.py)
- Test LOC: 24,904
- Hypothesis tests: 0
Reason or Problem
The geotiff module has ~25k LOC of tests against ~17k LOC of parser/writer code, but every test in
xrspatial/geotiff/tests/is example-based: handwritten inputs built withmake_minimal_tifffromconftest.py. Zero property-based or fuzz tests.Recent commits show a pattern of bugs that property-based testing catches cheaply:
babb72e_coords_to_transformbroke for 3D (y, x, band) DataArrays03d7380georef not inherited when reading overview IFDsc294578/dd907b8nodata sentinel poisoned cubic / COG overviews9a5f55eread_vrtleaked integer source nodata into a float-dtype VRTf0a09c0write_geotiff_gpu(file_like, cog=True)silently accepted an unsupported combination595ece8signature/docstring drift betweento_geotiff,write_vrt,write_geotiff_gpudbb5cebuser-defined CRS WKT not promoted to attrs on readcdf7d43LERC / JPEG 2000 decompression output not capped (block bomb)These cluster into a few classes:
Hand-written tests catch one variant at a time. Property tests cover the cross-product cheaply.
Proposal
Add Hypothesis-based property tests for the geotiff module.
Design:
A new test file
xrspatial/geotiff/tests/test_fuzz_hypothesis_<issue>.pywith three groups:Round-trip property. Strategies generate (width, height, dtype, compression, tiled, predictor, nodata), build a DataArray, write with
to_geotiff, read withopen_geotiff, assert array equality and attrs preservation. Bounded ranges (dims 1 to 32, a short dtype list, common codecs) so CI stays fast.IFD layout permutations. Use
make_minimal_tiffwith strategies over tag order, tile vs strip, big/little endian, with/without geo tags. Assertopen_geotiffeither returns a valid array or raises a typed exception from the geotiff module. Never a bareIndexError/struct.error/UnicodeDecodeError.Byte-level mutation fuzz. Take a valid TIFF from
make_minimal_tiff, flip one byte at a Hypothesis-chosen offset, assert the reader either parses consistently or raises a typed exception. Never silently returns wrong data, never segfaults.Each test uses
@settings(max_examples=50, deadline=None)to bound CI time.Hypothesis is not currently a test dependency. The implementation will guard the import with
pytest.importorskip('hypothesis')so the suite still runs when hypothesis is missing. A follow-up PR can add it tosetup.cfg [tests]once the harness has earned its keep.Usage:
pip install hypothesis pytest xrspatial/geotiff/tests/test_fuzz_hypothesis_*.py -vLocally,
--hypothesis-show-statisticsshows shrinking behaviour on failure.Value:
Stakeholders and Impacts
Geotiff module maintainers. No runtime behaviour changes, no new public API. Impact limited to the test suite.
Drawbacks
Alternatives
afl-tiff: heavier infrastructure, harder to run in CIUnresolved Questions
hypothesistosetup.cfg [tests]now or keep it opt-in? Defaulting to opt-in for this PR.Additional Notes or Context
Audit numbers:
xrspatial/geotiff/_*.py+__init__.py)