Skip to content

geotiff: extend backend parity matrix to 8 backends and 7 fixtures#2142

Merged
brendancol merged 3 commits into
mainfrom
issue-2132
May 19, 2026
Merged

geotiff: extend backend parity matrix to 8 backends and 7 fixtures#2142
brendancol merged 3 commits into
mainfrom
issue-2132

Conversation

@brendancol
Copy link
Copy Markdown
Contributor

Closes #2132.

Summary

  • Extends test_backend_parity_matrix.py from a 2-backend / 1-fixture scaffold to the full 8-backend / 7-fixture matrix described in geotiff: add formal backend parity test matrix #2132.
  • Adds an error-fixture sub-matrix (rotated ModelTransformationTag without allow_rotated) so the contract for raised exceptions has one home.
  • Backend / source-type compatibility lives on the _BackendSpec.compat set, so incompatible cells skip cleanly without taking the matrix down.

Backends covered

id dispatch source
numpy open_geotiff(path) local TIFF / VRT / HTTP / fsspec
dask+numpy open_geotiff(path, chunks=16) local TIFF / VRT / fsspec
gpu open_geotiff(path, gpu=True) local TIFF
dask+gpu open_geotiff(path, gpu=True, chunks=16) local TIFF
vrt-eager open_geotiff(path.vrt) local VRT
vrt-dask open_geotiff(path.vrt, chunks=16) local VRT
http-cog open_geotiff('http://...') local TIFF served via loopback HTTP
fsspec-memory open_geotiff('memory://...') local TIFF piped through fsspec

GPU rows skip when cupy or CUDA are missing. fsspec rows skip when fsspec is absent. HTTP rows reuse the loopback http.server pattern from test_cog_http_concurrent.py.

Fixtures covered

Out of scope

  • Writer parity (separate matrix in test_writer_matrix.py).
  • Pre-existing per-bug parity files (kept as named regression markers).
  • BytesIO source-type row: the fsspec memory:// row already covers the same code path for the matrix, and the existing test_bytesio_source.py keeps the explicit BytesIO behaviour pinned.

Test plan

  • pytest xrspatial/geotiff/tests/test_backend_parity_matrix.py -v reports 46 passed and 18 skipped (incompatible cells).
  • Existing per-bug parity files (test_backend_pixel_parity_matrix_1813.py, test_attrs_parity_1548.py, test_backend_kwarg_parity_1561.py, test_miniswhite_backend_parity_1797.py) stay green.
  • ruff and flake8 clean on the changed file.

…2132)

Builds on the matrix scaffold from #1985. Adds gpu, dask+gpu, vrt-eager,
vrt-dask, http-cog, and fsspec-memory backend rows, plus uint16
multiband tiled, float32 with nodata, int8 unmasked, COG, VRT mosaic,
and MinIsWhite fixtures. Adds an error-fixture sub-matrix for the
rotated ModelTransformationTag case.

Backend / fixture compatibility lives on the backend descriptor's
compat set, so incompatible (backend, source) pairs skip cleanly. HTTP
and fsspec rows reuse the patterns already in
test_cog_http_concurrent.py and test_features.py.
@github-actions github-actions Bot added the performance PR touches performance-sensitive code label May 19, 2026
Copy link
Copy Markdown
Contributor Author

@brendancol brendancol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review: geotiff: extend backend parity matrix to 8 backends and 7 fixtures

Blockers (must fix before merge)

None.

Suggestions (should fix, not blocking)

  • Dead label variable in the error test (xrspatial/geotiff/tests/test_backend_parity_matrix.py:1003-1011): the error matrix builds a label string and immediately dels it. The pytest parametrize id already carries that info. Drop the assignment, drop the del, drop the comment about tracebackhide-friendly messages. There is no assertion that uses label in this block; pytest.raises is doing the work.

  • Two of the VRT cells are duplicates (test_backend_parity_matrix.py:154-186): dask+numpy has _SRC_LOCAL_VRT in its compat set, so dask+numpy-vrt-mosaic runs with kwargs={"chunks": 16} against the VRT path. vrt-dask runs with the same kwargs against the same path. Same story for numpy-vrt-mosaic vs vrt-eager-vrt-mosaic (both kwargs={}). These cells exercise the same code. The issue's plan names vrt-eager and vrt-dask as the dedicated VRT rows, so the cleanest fix is to trim _SRC_LOCAL_VRT out of the numpy and dask+numpy compat sets and let the VRT-named backends own VRT.

Nits (optional improvements)

  • Type hints on _deliver_via_http / _deliver_via_fsspec (test_backend_parity_matrix.py:606-632): both functions are annotated as taking _FixtureSpec, but the error matrix passes _ErrorFixtureSpec instances. Duck typing on fix_id covers it. Widen the annotation to _FixtureSpec | _ErrorFixtureSpec or drop the constraint.

  • Session-scoped HTTP server keeps payloads forever (test_backend_parity_matrix.py:583-603): the payload_by_path dict on the shared handler is appended to per cell and never trimmed. Memory cost is tiny (each fixture is a few KB), but a per-test clear would be tidier. Skip if it never shows up as overhead.

What looks good

  • Backend and fixture descriptors are separated. Adding a new backend or fixture is a single row plus an optional builder, which is what the issue asked for.
  • The per-backend compat set is a good way to express (backend, source) incompatibility. Skip messages name the axis that cut the cell.
  • Pixel comparison is NaN-aware for float and byte-equal for int. dtype is checked against the spec rather than just the reference, so a uniform silent upcast across every backend still fails the cell.
  • masked_nodata is asserted on both the True (float32 with sentinel) and False (int8 with mask_nodata=False) sides. That locks both branches of the #2092 / #2127 contract.
  • The error sub-matrix mirrors the success matrix, so a new error fixture is one entry.
  • 46 cells pass and 18 skip cleanly (incompatible pairs). Existing per-bug parity files stay green.

Checklist

  • Algorithm matches the issue spec (#2132)
  • All implemented backends produce consistent results
  • NaN handling is correct (NaN-aware float comparison)
  • Edge cases covered (nodata sentinel, multiband, COG, MinIsWhite, rotated error)
  • N/A: no dask chunk-boundary logic in this PR (test harness only)
  • No premature materialization
  • N/A: no benchmark needed for a test harness
  • N/A: README feature matrix unchanged (no new public function)
  • Docstrings present and accurate

- Trim VRT compat from numpy / dask+numpy backends so vrt-eager and
  vrt-dask own the VRT row (drops two duplicate cells).
- Drop the dead ``label`` variable from the error sub-matrix.
- Widen the type hint on ``_deliver_via_http`` / ``_deliver_via_fsspec``
  so both ``_FixtureSpec`` and ``_ErrorFixtureSpec`` callers match.
- Clear the session HTTP server's payload dict between tests so the
  shared handler does not accumulate stale entries.
Copy link
Copy Markdown
Contributor Author

@brendancol brendancol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: review findings addressed

All four items from the previous review pass have been applied in df3b8eb:

  • Dead label variable: removed. The block now contains only pytest.raises and the inner read.
  • Duplicate VRT cells: _SRC_LOCAL_VRT is no longer in the compat sets for numpy and dask+numpy. The VRT fixture is now routed only through vrt-eager and vrt-dask, matching the issue's named-row layout. Cell count moves from 46 passed / 18 skipped to 44 passed / 20 skipped.
  • Type hints: _deliver_via_http and _deliver_via_fsspec are annotated as _FixtureSpec | _ErrorFixtureSpec so the error matrix call sites are covered by the signature.
  • HTTP payload accumulation: split the session-scoped server into _matrix_http_server_session (owns the socket + thread) and a function-scoped _matrix_http_server wrapper that clears payload_by_path before and after each test.

Local test run: 44 passed, 20 skipped on the matrix file; the per-bug regression markers (test_backend_pixel_parity_matrix_1813.py, test_attrs_parity_1548.py, test_backend_kwarg_parity_1561.py, test_miniswhite_backend_parity_1797.py) stay at 212 passed.

@brendancol brendancol merged commit d6acae8 into main May 19, 2026
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance PR touches performance-sensitive code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

geotiff: add formal backend parity test matrix

1 participant