geotiff: read external .tif.ovr sidecar overviews

[brendancol]

Closes #2112.

Summary

• Adds _sidecar.py for local-file sidecar discovery (<source>.ovr) and IFD parsing.
• Wires sidecar IFDs into read_to_array and _read_geo_info so overview_level >= 1 resolves the GDAL/rasterio convention where the base file holds level 0 and the sidecar holds the rest.
• Each sidecar IFD carries its origin (data, header) so strip / tile reads slice the right buffer.
• Removes the corpus-level skip for overview_external_ovr_uint16 on the eager numpy parity check.
• Cloud / HTTP / file-like sources skip the lookup and keep the base-only behaviour; sidecar support on those paths is follow-up.

Backend coverage

• numpy: covered (direct).
• cupy: not in scope; GPU reader walks the base IFD chain only. Corpus skip stays.
• dask+numpy: covered (per-chunk read goes through read_to_array).
• dask+cupy: not in scope; same reason as cupy.

Test plan

• test_sidecar_ovr_2112.py covers discovery (local / remote URIs / file-like / missing sidecar), parsing, end-to-end reads at each level, byte parity with rasterio, and the dask+numpy path.
• Golden corpus eager parity (test_golden_corpus_eager_numpy_1930.py) now exercises the sidecar overview levels.
• Full xrspatial/geotiff/tests/ suite: 4038 passed, 25 skipped.

[brendancol]

Self-review of PR #2114.

Coverage

• Local-file sidecar discovery only; cloud / HTTP / file-like sources skip the lookup (early return in find_sidecar).
• Each sidecar IFD is tagged with its origin (data, header) via object.__setattr__ so the strip / tile reader slices the right buffer.
• Both read_to_array and _read_geo_info see the merged pyramid, so the dask graph builder gets correct metadata for sidecar levels too. Dask + numpy chunks read through read_to_array, so dask + sidecar works without further changes.
• Byte parity check against rasterio for both sidecar levels of the bundled fixture passes.

Risk areas worth a closer look

• GPU and dask + GPU paths still walk the base IFD chain. The corpus skip stays for those two backends. The GPU reader's separate parse_all_ifds callers in _backends/gpu.py would need similar treatment to extend coverage.
• HTTP / cloud sidecar discovery is non-trivial (you would need to know the sidecar URL exists). Not in scope.
• The sidecar mmap is closed in a try / finally next to the base file close; if the base parse fails before sidecar load, the sidecar is None and skipped.

Cleanup

• Removed the _OVERVIEW_READER_GAPS["overview_external_ovr_uint16"] entry from the eager numpy parity test so the corpus exercises the sidecar path end-to-end.

[brendancol]

Follow-up to my self-review.

GPU and dask+GPU sidecar coverage -> deferred.
The GPU reader (_backends/gpu.py) and the dask+GPU pipeline both call parse_all_ifds directly. Threading the sidecar (data, header) swap through the nvCOMP tile decode would mean introducing a per-IFD source switch in the GPU kernel dispatcher, which is wider than this PR's scope. The corpus skip stays in place for those two backends and the gap is now narrowly described.

HTTP / cloud sidecar -> deferred.
The sidecar URL would need to be discoverable in the source's namespace (e.g. s3://bucket/path.tif.ovr exists alongside s3://bucket/path.tif). The _CloudSource layer already supports range reads but does not expose a "does object X exist" predicate without a HEAD round-trip. Leaving the discovery off entirely on remote URIs means the existing base-only behaviour is preserved and no spurious HEAD requests fire on every open.

Mmap close ordering -> verified.
Walked the failure modes:

• If parse_header(data) raises, sidecar is still None (the load is after parse), so the finally block only closes the base mmap.
• If find_sidecar returns a path but load_sidecar raises (e.g. malformed .ovr), the sidecar local stays None and the base mmap closes normally; the partial sidecar file handle was already closed inside load_sidecar's own try / finally.
• If attach_sidecar_origin raises (shouldn't, but worth checking), the sidecar local is set, so the outer finally closes the sidecar mmap too.

No change needed.

[brendancol]

Re-review after the GPU + HTTP + fsspec extension.

Suggestions (should fix, not blocking)

• HEAD probe and HTTP sidecar load bypass the existing urllib3 pool. _sidecar._probe_http uses urllib.request.Request(method="HEAD") and load_sidecar HTTP branch uses urllib.request.urlopen. The eager reader at _reader.py already maintains a _get_http_pool urllib3 PoolManager with connection reuse, retries, and a shared user agent. Sidecar discovery and load should share that pool so cloud reads of many sidecars do not re-open TCP per probe.

• _probe_http does not honour the private-host guard. _reader.py:556 rejects loopback / link-local / private hosts by default and only allows them with XRSPATIAL_GEOTIFF_ALLOW_PRIVATE_HOSTS=1. The new sidecar probe fires a HEAD against any URL the caller passes, so a user opening a public URL whose .ovr happens to live behind a private hostname (or whose 302 redirects to one) bypasses the guard. Route the probe through the same _validate_url_safe check the reader uses.

• GPU sidecar mmap is reaped by Python GC rather than explicitly closed. _backends/gpu.py documents this but Python is free to defer GC arbitrarily and the underlying file descriptor stays open until then. For long-running processes opening many GeoTIFFs in sequence this can accumulate file descriptors. A function-level try / finally that wraps both the metadata parse and the GPU decode would let close_sidecar run deterministically.

• SidecarOverviews is created and immediately mutated. attach_sidecar_origin writes back to the IFD instances via object.__setattr__. If a future caller reuses the same IFD list across requests, that mutation persists. Not a problem today, but a defensive option is to clone the IFD list (or capture the origin in a side dict) instead of mutating the parsed IFDs.

Nits (optional)

• SidecarOverviews.data: object could narrow to Union[mmap.mmap, bytes] so readers see the actual type.

• _is_fsspec_uri is duplicated in _sidecar.py to dodge the circular import. A shared helper module (or moving both definitions into _paths.py / similar) keeps the gates in sync.

• HEAD probe follows HTTP redirects by default. urllib.request.urlopen follows up to 10 redirects; a 302 from a public host to a private one would still satisfy the probe even after the safe-host fix above. Setting HTTPRedirectHandler to refuse or restrict redirects keeps the probe predictable.

• Comment in _backends/gpu.py:521-526 explains why the close is deferred, which is helpful, but the doc reads as if the lifetime equivalence with data (also touched after src.close()) is exact; it is approximate because data is a cached-mmap slice that survives release whereas the sidecar mmap is a fresh kernel mapping with its own lifecycle.

What looks good

• Discovery + load are layered: a cheap existence probe followed by an explicit load_sidecar, with consistent silent-fail-to-base-only semantics on every transport.
• GPU sidecar parity test (commit 55fb85c) confirms byte equality between CPU and GPU on both sidecar levels of the fixture.
• HTTP + fsspec tests use a loopback server / file:// URI so the coverage exercises real transport without external network dependency.
• close_sidecar helper centralises the close convention and keeps the call sites symmetric.

[brendancol]

Addressed the re-review (commit 722528e).

Item	Disposition
HEAD probe / HTTP load bypass urllib3 pool	Fixed -- both _probe_http and load_sidecar HTTP branch now route through _HTTPSource. Sidecar fetches reuse the eager reader's _get_http_pool urllib3 PoolManager, share retries, and inherit the manual redirect re-validation.
HEAD probe missing private-host guard	Fixed -- _HTTPSource.__init__ runs _validate_http_url (SSRF allow-list + IP pinning) before the connection opens. A loopback probe with no XRSPATIAL_GEOTIFF_ALLOW_PRIVATE_HOSTS=1 now returns None silently, matching the rest of the discovery contract. New regression test test_find_sidecar_http_probe_rejects_loopback_without_env_override pins this.
HEAD probe follows redirects implicitly	Closed by the _HTTPSource routing -- the source uses redirect=False and re-runs _validate_http_url on every Location header, so a 302 from a public host to a private one is rejected the same way the eager reader rejects it. Also swapped HEAD for a 1-byte Range GET so the probe works on CDNs / object stores with inconsistent HEAD support.
GPU sidecar mmap relied on GC	Fixed -- restructured read_geotiff_gpu so the GPU decode runs inside the same try/finally that owns the metadata parse. close_sidecar now runs deterministically after the decode finishes touching the buffer. File-descriptor count is bounded for long-running processes. The "this is approximate equivalence with data's lifetime" comment is gone with the GC path.
attach_sidecar_origin mutated IFDs via object.__setattr__	Fixed -- the helper now returns a dict[id(ifd), (data, header)] side map per call. Reader and GPU paths look up sidecar_origin.get(id(ifd), (data, header)) to resolve the buffer. No IFD instance is mutated; a future caller that caches parsed IFDs across requests inherits a clean object.
SidecarOverviews.data: object	Narrowed to SidecarBuffer = Union[mmap.mmap, bytes]. The new alias also annotates attach_sidecar_origin.
_is_fsspec_uri duplicated	Removed the local copy; _sidecar.py now imports the helper from _reader. _reader imports _sidecar lazily inside function bodies so the top-level reverse import does not form a cycle at module load.
Outdated GC-lifetime comment in _backends/gpu.py	Gone with the deterministic close.

Full xrspatial/geotiff/tests/ suite: 4046 passed, 25 skipped.