Skip to content

Phase 2 W3b — HF reverse proxy (SEC-02 / INVARIANT 2)#13

Merged
l17728 merged 21 commits into
mainfrom
feat/phase-2-w3b-hf-reverse-proxy
May 15, 2026
Merged

Phase 2 W3b — HF reverse proxy (SEC-02 / INVARIANT 2)#13
l17728 merged 21 commits into
mainfrom
feat/phase-2-w3b-hf-reverse-proxy

Conversation

@l17728
Copy link
Copy Markdown
Owner

@l17728 l17728 commented May 14, 2026

Summary

  • New controller-side reverse proxy GET /api/v1/hf-proxy/subtask/{id}: mTLS+JWT auth, a fail-closed verification chain (subtask exists → confused-deputy guard → assignment_token fence → epoch fence), HF URL reconstruction from server-side rows, tenant-token injection, streaming passthrough with a 5-header allowlist. Per-request httpx client closed on every path (success / HF transport error → 503 / other exception).
  • Executor no longer touches HF directly: ControllerClient.stream_hf (async-context-manager) is the only HF path; both downloaders (HfS3StreamDownloader, DirectOffsetDownloader) fetch through it; DirectOffsetDownloader._resolve_size became a bytes=0-0 range probe (proxy is GET-only). Assignment gained assignment_token.
  • ExecutorSettings.hf_token / hf_endpoint deleted; _io.make_http_client removed. New check_no_hf_token_in_executor invariant lint locks INVARIANT 2 for the executor package (with self-tests).
  • Zero schema changes / no alembic migration. No new runtime deps. New Settings.hf_proxy_timeout_seconds.

Spec: docs/superpowers/specs/2026-05-14-phase-2-w3b-hf-reverse-proxy-design.md
Plan: docs/superpowers/plans/2026-05-14-phase-2-w3b-hf-reverse-proxy.md

Test plan

  • uv run pytest tests/api/test_hf_proxy.py — 9 proxy cases (streaming, token injection, URL reconstruction, Range, 429, 401, 404, 403 NOT_YOUR_SUBTASK, 409 STALE_ASSIGNMENT / EPOCH_MISMATCH)
  • uv run pytest tests/executor/stream_hf headers, both downloaders rewired, _resolve_size range probe + fallback + RuntimeError branch, runner threads assignment_token
  • uv run pytest tests/e2e/test_executor_e2e.py — full executor→controller-proxy→HF(mock)→S3 path
  • uv run pytest tests/tools/test_lint_no_hf_token.py + uv run python tools/lint_invariants.py — the new lint passes on the production tree
  • uv run pytest -q — full suite green (235 passed, 1 deselected)

Known minor follow-ups (non-blocking, from final review)

  • make_fake_controller_client test double does not replicate the proxy's 5-header allowlist (latent — no current downloader reads a non-allowlisted header).
  • _resolve_size Content-Length fallback trusts the value without asserting status_code == 200 (HF is well-behaved here; downstream sha256 gate is fail-closed).

🤖 Generated with Claude Code

l17728 and others added 21 commits May 14, 2026 18:18
Controller-side GET /api/v1/hf-proxy/subtask/{id} streams HF files with the
tenant token injected server-side; executors stop calling HF directly and
lose ExecutorSettings.hf_token/hf_endpoint. Zero schema changes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
3 milestones: controller proxy endpoint, executor rewiring, integration
(lint + e2e + OpenAPI + docs + PR). TDD bite-sized steps, complete code.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…est (W3b)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… (W3b)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…W3b)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…lback (W3b)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… HF fields (W3b)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ook (W3b)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Reflects the corrections applied across the 9 implementation tasks:
test heartbeat-before-poll, proxy client cleanup + 503 mapping,
stream_hf return annotation, fake client X-Assignment-Token, the
_resolve_size RuntimeError test, and the lint full-text-scan note.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@l17728 l17728 merged commit 2924b6e into main May 15, 2026
12 checks passed
@l17728 l17728 deleted the feat/phase-2-w3b-hf-reverse-proxy branch May 15, 2026 01:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant