feat(router): add inference API translation between protocols

[pimlock]

Problem

Today the router requires an exact protocol match between the client SDK and the backend route. If an agent uses the OpenAI SDK (POST /v1/chat/completions), only routes with openai_chat_completions in their protocols list are eligible. Similarly, an Anthropic SDK client can only hit anthropic_messages routes.

This means:

• Route operators must provision backends matching every SDK their agents might use
• Agents are constrained to SDKs that match available backends
• Adding a new backend provider requires all agents to adopt its SDK, or the operator must run parallel routes

Proposed Solution

Add an API translation layer in the router so that a request detected as one protocol (e.g., openai_chat_completions) can be transparently translated and forwarded to a route speaking a different protocol (e.g., anthropic_messages), and the response translated back.

Key design considerations

• Route selection: When no exact-match route exists for the detected protocol, the router should attempt translation to a compatible protocol that does have a route
• Request translation: Convert request body schema between protocols (e.g., OpenAI messages array → Anthropic messages format, max_tokens semantics, tool/function calling schemas)
• Response translation: Convert response body back to the client's expected schema (e.g., Anthropic response → OpenAI ChatCompletion shape)
• Streaming: Handle SSE streaming translation (data: [DONE] vs Anthropic event types)
• Feature parity: Some features don't map 1:1 (e.g., OpenAI logprobs, Anthropic thinking). Define a strategy for unsupported features (error, best-effort, drop silently)
• Header translation: Already partially exists (auth header rewriting per protocol). Extend to cover other protocol-specific headers (e.g., anthropic-version)

Scope of initial translation pairs

At minimum:

• openai_chat_completions ↔ anthropic_messages

Nice to have:

• openai_responses → other protocols
• openai_completions → chat-based protocols

Where this fits

The translation should live in navigator-router, likely as a new module between detect_inference_pattern() (in navigator-sandbox) and proxy_with_candidates(). The router already rewrites headers and model IDs in backend.rs — translation extends this to full schema transformation.

Acceptance Criteria

• A client using the OpenAI SDK can be routed to an Anthropic-compatible backend (and vice versa) without the client changing SDKs
• Streaming responses are translated correctly
• Non-translatable features produce clear errors rather than silent data loss
• Existing exact-match routing is unaffected (no performance regression for same-protocol routes)
• Translation pairs are extensible for future protocols

[pimlock]

Exploration Notes: Architectural Approach

Intermediate Representation (IR) Pattern

Instead of building an N×M matrix of direct translators, use an IR pattern powered by the genai crate.

Flow:

1. Ingest: Accept raw JSON, deserialize using async-openai crate
2. Map to IR: Convert async-openai structs → genai::chat::ChatRequest / ChatMessage (handles structural normalization, e.g., extracting system prompts for Anthropic)
3. Execute: Pass IR to genai client, which handles provider-specific HTTP protocols
4. Map to Egress: Map genai response → async-openai response struct, serialize to JSON, return to client

Deterministic Routing (No Model-Name Guessing)

genai by default infers the provider from the model string (e.g., claude-3 → Anthropic). We bypass this since the same model name may exist across providers.

Solution: Configure genai::Client with a custom ModelMapper using a routing key syntax: "{provider_name}|{model_name}" (e.g., "anthropic|claude-3-5-sonnet-latest"). The mapper explicitly assigns the AdapterKind (protocol) and passes the raw model name to the provider.

Custom Endpoints

For routing to custom infrastructure (e.g., local vLLM cluster with OpenAI-compatible API), use genai's ServiceTargetResolver alongside the ModelMapper to dynamically rewrite Base URL and Auth headers per-request.

Open Implementation Items

• IR Mapper: Translation functions mapping async_openai::types::ChatCompletionRequestMessage variants → genai::chat::ChatMessage
• SSE Streaming (high complexity): When stream: true, consume genai's ChatStreamEvent, map to async_openai::types::CreateChatCompletionStreamResponse, push to client via SSE
• Provider-Specific Quirks ("lossy" edge cases): Strategy for features that don't map cleanly (e.g., Anthropic's explicit prompt caching cache_control). Decide between auto-injection, stripping, or lowest-common-denominator API