feat: SSM zero-on-realloc guard + sentinel ablation harness

[Natfii]

Summary

Three-commit series adding an SSM zero-on-realloc guard alongside the existing full-attention KV zero-on-realloc path, plus a reproducible sentinel-gated 4-arm ablation harness and the 2026-05-15 harness-validation evidence.

• Commit 1 — production patch (633a96669): MambaBlockZeroer sister to KVBlockZeroer. Walks the same block-ID list at request-free / block-realloc time, zeroing recycled conv_state / ssm_state rows via torch.index_fill_ before the next prefill writes them. Always-on; rollback is git revert (EngineCore strips env vars, so an env kill switch is unreliable).
• Commit 2 — ablation harness (655fc8c32): scripts/ablation/ (runner + compare + overlay patch + prep helper), docs/research/2026-05-15-ssm-zero-on-realloc/README.md, plus backwards-compatible instrumentation in scripts/gsm8k_eval_50.py (--run-index, --metrics-url, per-Q JSONL, /metrics snapshots).
• Commit 3 — evidence (8f16a9eb1): benchmarks/nvllm/traces/ssm_zero_on_realloc/2026-05-15-sentinel-ablation/ with summary.md per AGENTS.md §4, ANALYSIS.md, runner_manifest.json, comparison.json, four per-arm verdicts. Also fixes a residual IMAGE_DIGEST emit bug in the runner (cmd || echo X -> cmd || true + ${VAR:-X}).

Why this is shipped (and what is NOT claimed)

The existing KVBlockZeroer (upstream PR vllm-project#35219) clears full-attn KV blocks but skips Mamba layers because the conv / ssm page sizes differ from the full-attn page size and cannot share the Triton kernel's uniform PAGE_SIZE_EL. MambaBlockZeroer covers the remaining state. This addresses the mamba-state lifecycle gap identified during the β-coop sustained-load collapse diagnosis arc.

Honest framing — what this PR does and does not prove:

• The β-coop sustained-load collapse was not reproducing on the 2026-05-15 host. The 4-arm sentinel ablation in this PR cannot prove the SSM patch fixes the collapse — it proves the sentinel-gating mechanism works and that the patch is correctness/perf neutral under non-collapse load. The original fix-validation evidence (patched soak: 48/48/48/48/48 vs D2.7 baseline 47/47/47/11/35; decode ~2.5 → ~9.4 tok/s) lives in the local /tmp/ssm_zero_on_free_soak/ directory and is referenced in memory:project_beta_coop_sustained_collapse. The committed harness is for the next collapse discrimination window.
• No perf claim. No nsys trace was captured. Median decode is flat across all 4 arms (within 0.03 tok/s).
• KV new_block_ids channel relax is NOT shipped to default. The 4-arm sweep showed a deterministic -1 question on the kv_only arm (47/50 × 5 vs 48/50 × 5 on the other three arms). The relax is kept in the harness overlay patch only.

4-arm sentinel ablation result (2026-05-15)

Arm	SSM sentinel	KV sentinel	runs (correct/50)	first_fire (ssm,kv)	gate_pass	harness_pass
both	1	1	48,48,48,48,48	(1, 1)	true	true
neither	0	0	48,48,48,48,48	(0, 0)	true	true
ssm_only	1	0	48,48,48,48,48	(1, 0)	true	true
kv_only	0	1	47,47,47,47,47	(0, 1)	true	true

harness_pass=true for all four arms means: when SSM_sentinel=1 the SSM gate fired (and not when 0); same for KV. The env-strip confound that made a prior env-gated attempt a null A/B is eliminated. See benchmarks/nvllm/traces/ssm_zero_on_realloc/2026-05-15-sentinel-ablation/summary.md for the full host/image manifest and the per-Q breakdown.

Test plan

• Production patch diff inspected; no sentinel / env / debug residue (grep -c "sentinel\|SENTINEL\|/run/nvllm" vllm/v1/worker/utils.py vllm/v1/worker/gpu_model_runner.py → 0).
• Sentinel overlay patch roundtrips: git apply scripts/ablation/ssm_sentinel_overlay.patch then git checkout -- restores HEAD cleanly; markers _SSM_ZERO_SENTINEL and _KV_ZERO_SENTINEL land where expected.
• Runner script bash-syntax checked (bash -n); compare script Python-syntax checked (ast.parse).
• scripts/gsm8k_eval_50.py backwards-compatible: --run-index defaults to 0, --metrics-url defaults to None; existing callers are unaffected.
• All four per-arm verdict.json files parse and report gate_pass=true + harness_pass=true.
• Future: re-run the harness against a collapsing host state to discriminate which half of the original /tmp/ssm_zero_on_free_soak patch was load-bearing (working hypothesis: SSM-only).
• Future: phaseE-tax bench re-run post-SSM-fix to verify the Phase 4 verdict still holds (task Support various block sizes vllm-project/vllm#38, separate).

Constraints (from work-session policy)

• AI assistance was used for the patch authorship, the harness, and the commit-message drafting. Human authored the design constraints (drop KV channel relax from production, no "fixes collapse" claim, no perf claim without nsys), reviewed every changed line, and ran the ablation suite end-to-end.
• This PR targets Navi-AI-Lab/nvllm main, not upstream vllm-project/vllm, per memory:feedback_never_touch_upstream_vllm.

🤖 Generated with Claude Code