glcm: unbounded window_size and distance enable CPU and memory DoS

[brendancol]

Describe the bug

glcm_texture() in xrspatial/glcm.py validates window_size only as >= 3 and distance only as >= 1, with no upper bound on either. The numba kernel iterates the full window_size x window_size neighborhood for every pixel regardless of raster shape. The interior bounds check on (wy, wx, ny, nx) drops invalid pairs inside the loop, but the range(r - half, r + half + 1) still runs window_size^2 iterations per pixel either way.

One int from the caller is enough to hang the host:

• window_size=1_000_001 on a 10x10 raster runs ~10^12 inner iterations per pixel times 100 pixels, so ~10^14 loop iterations where every neighbor fails the bounds test. The numba loop runs for hours at 100% CPU.
• distance=10**9 is accepted with no cap. dy *= distance and dx *= distance can overflow int64 in the JIT-compiled kernel, and even when they don't, the shifted pair runs off the raster every time so no pair is ever valid.
• On the dask backends, depth = window_size // 2 + distance is passed to map_overlap. A huge window_size pads every block to roughly (chunk_h + window_size, chunk_w + window_size) float64, which is a memory DoS.

Expected behavior

Reject window_size and distance values that cannot produce any valid pixel pairs on the input raster:

• window_size <= min(rows, cols) (a window larger than the raster cannot sample anything new)
• distance <= window_size // 2 (a pair separated by more than half the window cannot both fit in the window)

Raise ValueError naming the offending value and the cap.

Reproduction

import numpy as np
import xarray as xr
from xrspatial import glcm_texture

agg = xr.DataArray(np.zeros((10, 10)), dims=['y', 'x'])
glcm_texture(agg, window_size=1_000_001)  # runs indefinitely

Affected backends

numpy, cupy, dask+numpy, dask+cupy. The check lives in the public entrypoint, so one cap covers every backend. The cupy and dask+cupy paths call through to the numba CPU kernel after a cupy.asnumpy transfer, so they inherit the same DoS.

Severity

HIGH. One int to the public API can exhaust host CPU (and host memory on dask paths).

Category

Cat 1 (Unbounded allocation / DoS) per the security sweep.

Additional context

Found during the glcm security sweep on 2026-04-24. levels is already capped at 256, so the per-pixel np.zeros((levels, levels)) allocation inside the kernel is safe. One-fix-per-PR convention applies. Prior related fixes: bilateral (#1236), convolution (#1241), contour (#1240), bump (#1231).