Guard watershed_dinf() against unbounded memory allocations (#1345)#1348
Merged
Conversation
Mirrors the pattern from watershed_d8 (#1330) and watershed_mfd (#1342). CPU peak is 33 B/pixel (single-channel float64 angle, same as D8). GPU peak is 24 B/pixel (caller's flow_dir + pour_points + result). Guards run on numpy/cupy dispatch paths; dask paths skip them since per-tile allocations are bounded by chunk size.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
_BYTES_PER_PIXEL = 33(CPU) and_GPU_BYTES_PER_PIXEL = 24(GPU) helpers and guards the publicwatershed_dinfdispatch on the eager numpy and cupy branches. Dask branches skip the guard.Bytes per pixel
CPU: fd cast (8) + labels (8) + state int8 (1) + path_r int64 (8) + path_c int64 (8) = 33 B/px.
GPU (cupy path copies the device input to host, runs the CPU kernel, ships the result back): caller's flow_dir (8) + caller's pour_points (8) + final cp.asarray(out) (8) = 24 B/px on device.
D-inf carries a single real-valued angle, so the CPU footprint matches D8 rather than MFD's 8-channel fractions buffer.
Test plan
test_watershed_dinf.py::TestMemoryGuard:test_numpy_huge_raster_raisespatches available memory to 1 byte and expects MemoryError.test_numpy_normal_input_succeedsruns a 10x10 raster through the guard with real RAM.test_dask_path_skips_guardconfirms the dask backend bypasses the guard even when memory is patched to 1 byte.test_error_message_mentions_dimensionschecks the error string includes grid shape and the "dask" hint.test_cupy_huge_raster_raisesis gated on@cuda_and_cupy_availableand patches GPU memory to 1 byte.pytest xrspatial/hydro/tests/test_watershed_dinf.py: 19/19 pass.test_basin_dask_temp_cleanup.