bug(mcp/sse): /messages?sessionId= returns 404 — all MCP tool calls non-functional on pi.ruv.io

[RobLe3]

Summary

All MCP tool calls via the SSE transport on pi.ruv.io return HTTP 404. The SSE connection establishes successfully and returns a valid session endpoint, but POSTing to that endpoint fails. The REST API (/v1/...) is unaffected and fully functional.

Environment

• Endpoint: https://pi.ruv.io/sse (hosted instance)
• Transport: MCP SSE (as configured in ~/.claude/settings.json → mcpServers.pi-brain)
• Client: Claude Code (Sonnet 4.6) — deferred tool schema for all 37 brain_* tools
• Observed: 2026-03-28 (multiple attempts across several hours)

Steps to Reproduce

# Step 1 — SSE connection: WORKS (returns session endpoint)
curl -s "https://pi.ruv.io/sse" \
  -H "Authorization: Bearer $BRAIN_API_KEY" \
  -H "Accept: text/event-stream" \
  --max-time 3

# Output:
# event: endpoint
# data: /messages?sessionId=ba8abff6-cb7a-4790-9f84-4a14fdeab867

# Step 2 — POST tool call to session endpoint: FAILS with 404
curl -s -w "\nHTTP: %{http_code}" \
  -X POST "https://pi.ruv.io/messages?sessionId=ba8abff6-cb7a-4790-9f84-4a14fdeab867" \
  -H "Authorization: Bearer $BRAIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"brain_status","arguments":{}}}' \
  --max-time 8

# Output: HTTP: 404

# Step 3 — REST API: WORKS fine
curl -s "https://pi.ruv.io/v1/status" \
  -H "Authorization: Bearer $BRAIN_API_KEY"

# Output: {"total_memories":5190,"total_contributors":97,...} HTTP: 200

Observed vs Expected

Endpoint	Expected	Actual
GET /sse	200, streams endpoint event	✅ 200, event received
POST /messages?sessionId=<id>	200, JSON-RPC response	❌ 404
POST /messages (no session)	405	✅ 405 (route exists)
GET /v1/status	200	✅ 200

Suspected Root Cause

Likely a session affinity / distributed state issue introduced during or after the Cloud Run redeployment that fixed issue #305.

The MCP SSE transport works as follows:

1. Client connects to /sse → server registers an in-memory session, streams endpoint event
2. Client POSTs JSON-RPC messages to /messages?sessionId=<id>

If Cloud Run routes the SSE connection to Instance A (session registered there) but routes the subsequent POST to Instance B (no knowledge of that session), the POST gets 404.

Issue #305 noted session affinity: enabled — if the recent redeployment (ruvbrain-00159-cjl) reset or misconfigured session affinity, this would explain the symptom exactly.

Alternative: The session TTL may have been shortened to < the roundtrip time for a client to receive the SSE event and send the first POST.

Impact

• All 37 brain_* MCP tools are non-functional on the hosted pi.ruv.io instance
• Any Claude Code / MCP client configured to use https://pi.ruv.io/sse cannot call any tools
• The REST API is a functional workaround but requires direct curl calls — not usable from MCP clients

Collective Stats at Time of Report

The underlying data is healthy and growing:

• 5,190 memories (up from 3,154 in late March)
• 8,694,179 graph edges (up from 2,758,924)
• LoRA epoch 86 (up from 59)
• avg_quality: 0.754

Suggested Fix

1. Verify Cloud Run session affinity is still enabled on the current revision (ruvbrain-00159-cjl) — check --session-affinity flag
2. If affinity is on: Check whether session state is correctly restored after the restart — in-memory sessions would not survive a restart even with affinity
3. Longer term: Move session state to a persistent store (Redis or the existing ruvector DB) so sessions survive instance restarts and multi-instance routing

Related

• pi.ruv.io: cognitive cycle blocks HTTP handler causing 504 upstream timeouts #305 — pi.ruv.io cognitive cycle blocking HTTP handlers (504 timeouts, redeployment fix) — the redeployment that fixed pi.ruv.io: cognitive cycle blocks HTTP handler causing 504 upstream timeouts #305 may have introduced this regression

[ruvnet]

Root Cause Confirmed

In-memory session storage on multi-instance Cloud Run.

The MCP SSE handler stores sessions in a process-local DashMap:

// routes.rs:182-183
let sessions: Arc<dashmap::DashMap<String, tokio::sync::mpsc::Sender<String>>> =
    Arc::new(dashmap::DashMap::new());

When Cloud Run scales to >1 instance:

1. GET /sse hits Instance A → session abc123 registered in A's DashMap
2. POST /messages?sessionId=abc123 hits Instance B → sessions.get("abc123") returns None → 404

Fix Options (ordered by complexity)

Option 1: Force single instance (quick fix, limits scaling)

gcloud run services update ruvbrain --max-instances=1 --region=us-central1

Option 2: Enable session affinity (recommended)

gcloud run services update ruvbrain --session-affinity --region=us-central1

This sends a __session_affinity cookie that routes subsequent requests to the same instance. The SSE client needs to pass cookies on the POST.

Option 3: External session store (production-grade)
Replace DashMap with Redis/Memorystore so sessions are shared across instances. More complex but enables horizontal scaling.

Verification

# Check current session affinity setting
gcloud run services describe ruvbrain --region=us-central1 \
  --format="value(spec.template.metadata.annotations['run.googleapis.com/sessionAffinity'])"

The reporter is correct — this was likely caused by the redeployment for #305.

[ruvnet]

Fixed

Root cause: Session affinity was enabled but Cloud Run had maxScale=10. SSE clients (including MCP EventSource) don't forward the affinity cookie on the POST to /messages, so the POST could hit a different instance that doesn't have the session.

Fix applied: Set max-instances=1 on ruvbrain service.

gcloud run services update ruvbrain --max-instances=1 --region=us-central1

New revision: ruvbrain-00163-r2s (100% traffic)

Validation:

$ curl -s "https://pi.ruv.io/sse" → Session ID: b3d00d2e-...
$ POST /messages?sessionId=b3d00d2e-... → HTTP 202 ✅
$ POST brain_status tool call → HTTP 202 ✅

Both SSE connection and MCP tool calls now work end-to-end.

Note: This limits the service to 1 instance (containerConcurrency=20 still allows 20 concurrent requests). For proper multi-instance scaling, the session store should be moved from in-memory DashMap to Redis/Memorystore (tracked separately). Existing MCP clients may need to reconnect to pick up a new session on the single instance.