feat: replay 9 upstream features from closed-not-merged PRs

[KooshaPari]

Summary

Replays 9 upstream features from closed-not-merged PRs that were never integrated into main:

• fix(responses): prevent JSON tree corruption from literal control chars in function output router-for-me/CLIProxyAPI#1672: Prevent JSON tree corruption from literal control chars in function output
• fix(auth): limit auto-refresh concurrency to prevent refresh storms router-for-me/CLIProxyAPI#1686: Limit auto-refresh concurrency to prevent refresh storms (5s interval + semaphore)
• fix: correct Gemini API schema parameter naming router-for-me/CLIProxyAPI#1648: Correct Gemini API schema parameter naming (parametersJsonSchema -> parameters)
• feat: add official Termux (aarch64) build to release workflow router-for-me/CLIProxyAPI#1233: Add official Termux (aarch64) build to release workflow
• Fix duplicate/empty tool_use blocks in OpenAI->Claude streaming translation router-for-me/CLIProxyAPI#1579: Fix duplicate/empty tool_use blocks in OpenAI->Claude streaming translation
• feat(translator): OpenAI web search annotations passthrough router-for-me/CLIProxyAPI#1539: Pass through OpenAI web search annotations (url_citation) to all formats
• feat: add sticky-round-robin routing strategy router-for-me/CLIProxyAPI#1673: Add sticky-round-robin routing strategy with X-Session-Key support
• fix(antigravity): keep primary model list and backfill empty auths router-for-me/CLIProxyAPI#1699: Keep primary antigravity model list and backfill empty auths with cached fallback
• fix(iflow): improve 406 handling, stream stability, and auth availability router-for-me/CLIProxyAPI#1650: Harden iflow 406 retry, stream fallback, and auth availability

Not included (needs further work)

• fix(auth): don't cool down keys on count_tokens 4xx router-for-me/CLIProxyAPI#1599 (count_tokens 4xx no cooldown): conflicts with executeWithFallback refactor, needs manual integration

Already in main (verified)

• fix(antigravity): place tool_result images in functionResponse.parts and unify mimeType router-for-me/CLIProxyAPI#1682 (tool result image/mimeType)
• fix: correct Gemini API schema parameter naming router-for-me/CLIProxyAPI#1648 gemini schema (partially)
• feature(ampcode): Improves AMP model mapping with alias support router-for-me/CLIProxyAPI#1314 (ampcode alias model mapping)
• Feat(vertex): add prefix field router-for-me/CLIProxyAPI#1302 (vertex prefix field)

Test plan

• Verify Go build passes
• Run existing test suite
• Verify import paths are all kooshapari/cliproxyapi-plusplus
• Test sticky-round-robin with X-Session-Key header
• Test antigravity model fallback when fetch fails

🤖 Generated with Claude Code

Summary by CodeRabbit

Release Notes

New Features

• Added sticky round-robin routing strategy for session-aware load balancing
• Enabled automatic model backfilling across Antigravity providers
• Session key header propagation support

Improvements

• Enhanced tool parameter compatibility across Gemini, Claude, and OpenAI formats
• Integrated web search annotations into translations for all target formats
• Implemented request retry logic and diagnostic logging for upstream failures

Reliability

• Added in-memory caching for Antigravity models with fallback mechanisms
• Reduced refresh check interval to 5s with concurrent rate limiting (16 parallel)
• Implemented alternative authentication fallback for transient errors

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 04a260f and 02fd192.

📒 Files selected for processing (25)

• .github/workflows/release.yaml
• internal/api/handlers/management/config_basic.go
• internal/config/config.go
• internal/runtime/executor/antigravity_executor.go
• internal/runtime/executor/antigravity_executor_models_cache_test.go
• internal/runtime/executor/iflow_executor.go
• internal/runtime/executor/iflow_executor_test.go
• internal/translator/antigravity/gemini/antigravity_gemini_request.go
• internal/translator/antigravity/gemini/antigravity_gemini_request_test.go
• internal/translator/gemini/openai/responses/gemini_openai-responses_request.go
• internal/translator/openai/claude/openai_claude_response.go
• internal/translator/openai/claude/openai_claude_response_test.go
• internal/translator/openai/gemini/openai_gemini_response.go
• internal/translator/openai/openai/responses/openai_openai-responses_request.go
• internal/translator/openai/openai/responses/openai_openai-responses_response.go
• sdk/api/handlers/handlers.go
• sdk/cliproxy/auth/conductor.go
• sdk/cliproxy/auth/conductor_overrides_test.go
• sdk/cliproxy/auth/selector.go
• sdk/cliproxy/auth/selector_test.go
• sdk/cliproxy/builder.go
• sdk/cliproxy/service.go
• sdk/cliproxy/service_antigravity_backfill_test.go
• test/builtin_tools_translation_test.go
• test/openai_websearch_translation_test.go

---

📝 Walkthrough

Walkthrough

This PR introduces sticky round-robin routing strategy with session-based stickiness, implements request-level caching for antigravity models with fallback behavior, enhances IFlow executor with retry logic and request signing, adds auth fallback mechanisms with concurrency controls, and expands translation pipelines to propagate web-search annotations and citations across Claude, Gemini, and OpenAI formats.

Changes

Cohort / File(s)	Summary
Build & Release .github/workflows/release.yaml	Adds new GitHub Actions job to build Termux ARM64 binaries via Docker, computing version/commit/build-date metadata and creating archived artifacts for release distribution.
Routing Configuration internal/config/config.go, internal/api/handlers/management/config_basic.go, sdk/cliproxy/builder.go, sdk/cliproxy/service.go	Extends routing strategy support to recognize "sticky-round-robin" (and aliases "stickyroundrobin", "srr") across configuration, handler normalization, and service/builder instantiation layers.
Sticky Round-Robin Selector sdk/cliproxy/auth/selector.go, sdk/cliproxy/auth/selector_test.go	Introduces new StickyRoundRobinSelector with session-key based routing, sticky session-to-auth mappings with memory eviction, and improved error handling for unavailable auth scenarios; adds test validating service unavailable responses.
Auth Conductor & Fallback sdk/cliproxy/auth/conductor.go, sdk/cliproxy/auth/conductor_overrides_test.go	Reduces refresh check interval from 30s to 5s, adds concurrency-limited refresh via semaphore (max 16 concurrent), introduces alternative auth detection for transient errors (408/500/502/503/504), standardizes "no auth available" error responses, and adds tests validating cooldown behavior with/without alternatives.
Antigravity Model Caching internal/runtime/executor/antigravity_executor.go, internal/runtime/executor/antigravity_executor_models_cache_test.go	Implements in-memory cache for antigravity models with thread-safe load/store accessors, deep cloning on retrieval, fallback on all error paths, and comprehensive tests validating cache semantics and immutability.
IFlow Executor Refactoring internal/runtime/executor/iflow_executor.go, internal/runtime/executor/iflow_executor_test.go	Major refactoring centralizing HTTP execution, adding idempotency/conversation metadata keys, implementing 406 retry with optional request signing, capturing request diagnostics (session/conversation IDs), parsing business-status errors, synthesizing SSE chunks from non-streaming responses, and logging rejected request diagnostics; includes extensive test coverage for retry, diagnostics, streaming fallback, and error propagation.
Gemini API Compatibility internal/translator/antigravity/gemini/antigravity_gemini_request.go, internal/translator/antigravity/gemini/antigravity_gemini_request_test.go	Handles function_declarations by renaming parametersJsonSchema to parameters when parameters is absent, preserving API compatibility; adds tests validating both parameters and parametersJsonSchema paths.
Gemini-OpenAI Response Handling internal/translator/gemini/openai/responses/gemini_openai-responses_request.go	Adds control-character detection helper and conditional JSON embedding for function call outputs to prevent sjson.SetRaw corruption when outputs contain literal control characters.
Claude Response Translation internal/translator/openai/claude/openai_claude_response.go, internal/translator/openai/claude/openai_claude_response_test.go	Introduces canonical tool name mapping and annotation accumulation, adds ToolCallAccumulator.Started tracking to emit content_block_start only once per tool call, normalizes tool names in both streaming/non-streaming paths, propagates web-search annotations as citations; includes tests validating tool name canonicalization and content-block emission.
Gemini Response Translation internal/translator/openai/gemini/openai_gemini_response.go	Adds annotation accumulation from OpenAI responses and attaches as groundingMetadata with citations array in both streaming and non-streaming paths.
OpenAI Request/Response Translation internal/translator/openai/openai/responses/openai_openai-responses_request.go, internal/translator/openai/openai/responses/openai_openai-responses_response.go	Changes tool pass-through behavior to preserve builtin tools (previously filtered); accumulates web-search annotations into AnnotationsRaw and propagates to response content parts, output items, and text messages.
HTTP Request Metadata sdk/api/handlers/handlers.go	Forwards X-Session-Key header into execution metadata under "session_key" key for request tracing and session-based routing.
Antigravity Model Backfilling sdk/cliproxy/service.go, sdk/cliproxy/service_antigravity_backfill_test.go	Adds backfill mechanism to propagate primary models from source antigravity auth to other clients lacking models, applying auth_kind derivation, model exclusions, and aliases; includes tests validating model registration and exclusion handling.
Translation & Tool Tests test/builtin_tools_translation_test.go, test/openai_websearch_translation_test.go	Renames and updates tool translation tests; adds comprehensive web-search translation suite covering OpenAI↔Claude/Gemini/Responses formats with annotation/citation/groundingMetadata propagation in both streaming and non-streaming paths.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~75 minutes

Possibly related PRs

• ci: sync branch feat/clean-base-1698-strip-empty-messages-rebase #517: Modifies the same auth conductor (sdk/cliproxy/auth/conductor.go) with overlapping refresh-concurrency and auto-refresh behavior changes.

Poem

🐰 A sticky hop through session lanes,
Where models cache and auth remains,
Retry with grace, when signatures sign,
Web-search citations shine and align,
Round-robin routes no longer plain! 🌟

✨ Finishing Touches

• 📝 Generate docstrings (stacked PR)
• 📝 Generate docstrings (commit on current branch)

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch upstream-feature-replay

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.5.0)

Error: can't load config: the Go language version (go1.25) used to build golangci-lint is lower than the targeted Go version (1.26.0)
The command is terminated due to an error: can't load config: the Go language version (go1.25) used to build golangci-lint is lower than the targeted Go version (1.26.0)

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request integrates several previously unmerged upstream features, enhancing the API's robustness, performance, and compatibility across various LLM providers. Key improvements include better handling of JSON data integrity, optimized resource management for auto-refresh, standardized API parameter naming, and more intelligent routing and error recovery mechanisms. These changes collectively improve the stability and feature parity of the system.

Highlights

• JSON Tree Corruption Prevention: Implemented a fix to prevent JSON tree corruption when literal control characters are present in function output, specifically in OpenAI to Gemini response translation.
• Auto-Refresh Concurrency Limiting: Introduced a semaphore to limit the concurrency of auto-refresh operations to 16, along with reducing the refresh interval to 5 seconds, to prevent refresh storms.
• Gemini API Schema Parameter Naming Correction: Corrected the Gemini API schema parameter naming, normalizing 'parametersJsonSchema' to 'parameters' for consistency.
• Sticky Round-Robin Routing Strategy: Added a new 'sticky-round-robin' routing strategy that routes requests with the same 'X-Session-Key' header to the same backend credential, falling back to standard round-robin if no session key is present.
• Duplicate/Empty Tool Use Blocks Fix: Addressed issues with duplicate or empty tool_use blocks in OpenAI to Claude streaming translation, ensuring tool calls are emitted correctly and only once.
• OpenAI Web Search Annotations Passthrough: Enabled the passthrough of OpenAI web search annotations (url_citation) to all supported output formats (Claude, Gemini, and OpenAI Responses).
• Antigravity Model List Backfill: Implemented caching for the primary Antigravity model list and a mechanism to backfill empty Antigravity authentications with this cached fallback list when fetching models fails.
• iFlow 406 Retry and Stream Fallback Hardening: Improved the iFlow executor to retry requests without a signature if an initial request returns a 406 Not Acceptable status. Also added fallback logic for streaming requests that receive non-SSE JSON responses.
• Auth Availability and Cooldown Logic: Refined the authentication cooldown logic such that transient errors (e.g., 500s) will not trigger a cooldown if no alternative authentication is available for the requested model.

Changelog

• internal/api/handlers/management/config_basic.go
  • Updated import paths to the new repository location
  • Added 'sticky-round-robin' as a recognized routing strategy
• internal/config/config.go
  • Updated import paths to the new repository location
  • Updated RoutingConfig documentation to include 'sticky-round-robin' strategy
• internal/runtime/executor/antigravity_executor.go
  • Updated import paths to the new repository location
  • Introduced antigravityPrimaryModelsCache for caching fetched model lists
  • Added helper functions (cloneAntigravityModels, storeAntigravityPrimaryModels, loadAntigravityPrimaryModels, fallbackAntigravityPrimaryModels) for cache management
  • Modified FetchAntigravityModels to use the cached model list as a fallback on token errors, request creation errors, network errors, or unexpected HTTP statuses
• internal/runtime/executor/antigravity_executor_models_cache_test.go
  • Added new test file for Antigravity primary models cache functionality
  • Included tests for ensuring empty model lists do not overwrite the cache and that loaded models are clones to prevent mutation
• internal/runtime/executor/iflow_executor.go
  • Updated import paths to the new repository location
  • Added iflowRequestAttempt struct to snapshot request details for logging
  • Refactored request execution into executeChatCompletionsRequest to handle 406 Not Acceptable retries without signature
  • Introduced iflowRequestIDs to extract session and conversation IDs from metadata or payload
  • Implemented synthesizeOpenAIStreamingChunksFromNonStream to convert non-SSE JSON responses into streaming chunks for fallback
  • Added parseIFlowBusinessStatusError, parseNumericStatus, normalizeIFlowBusinessStatus for robust business error parsing
  • Added parseOpenAIStreamNetworkErrorWithoutContent to detect network errors in streaming responses without content
• internal/runtime/executor/iflow_executor_test.go
  • Added tests for executeChatCompletionsRequest to verify retry behavior on 406 status codes
  • Included tests for diagnostic logging on 403 Forbidden responses and absence of diagnostics on 500 errors
  • Added tests for streaming behavior, including emitting message_stop and falling back when upstream returns non-SSE JSON
• internal/translator/antigravity/gemini/antigravity_gemini_request.go
  • Updated import paths to the new repository location
  • Corrected Gemini API schema parameter naming from parametersJsonSchema to parameters within tool declarations
• internal/translator/antigravity/gemini/antigravity_gemini_request_test.go
  • Added tests to ensure Gemini tool parameters keys are preserved or normalized correctly
• internal/translator/gemini/openai/responses/gemini_openai-responses_request.go
  • Added containsLiteralControlChars function to check for literal control characters in JSON output
  • Modified function output handling to prevent JSON tree corruption by safely escaping values with literal control characters
• internal/translator/openai/claude/openai_claude_response.go
  • Updated import paths to the new repository location
  • Added AnnotationsRaw field to ConvertOpenAIResponseToAnthropicParams to accumulate web search annotations
  • Added CanonicalToolNameByLower to ConvertOpenAIResponseToAnthropicParams for tool name canonicalization
  • Modified streaming and non-streaming translation to handle OpenAI web search annotations as Claude citations
  • Improved tool call handling in streaming to emit tool_use start only once and canonicalize tool names
• internal/translator/openai/claude/openai_claude_response_test.go
  • Added new test file for Claude response translation, specifically for tool name canonicalization and single emission of tool start events
• internal/translator/openai/gemini/openai_gemini_response.go
  • Added AnnotationsRaw field to ConvertOpenAIResponseToGeminiParams to accumulate web search annotations
  • Modified streaming and non-streaming translation to handle OpenAI web search annotations as Gemini grounding metadata
• internal/translator/openai/openai/responses/openai_openai-responses_request.go
  • Modified ConvertOpenAIResponsesRequestToOpenAIChatCompletions to pass through built-in tools (e.g., web_search) instead of ignoring them
• internal/translator/openai/openai/responses/openai_openai-responses_response.go
  • Added AnnotationsRaw field to oaiToResponsesState to accumulate web search annotations
  • Modified streaming and non-streaming translation to populate OpenAI Responses with web search annotations
• sdk/api/handlers/handlers.go
  • Updated import paths to the new repository location
  • Added logic to forward the X-Session-Key header as metadata for sticky routing
• sdk/cliproxy/auth/conductor.go
  • Updated import paths to the new repository location
  • Reduced refreshCheckInterval to 5 seconds and added refreshMaxConcurrency (16) to limit concurrent refresh operations
  • Introduced refreshSemaphore to enforce concurrency limits on auth refreshes
  • Modified MarkResult to prevent cooldowns for transient errors if no alternative auth is available for the model
  • Added hasAlternativeAuthForModelLocked to check for available alternative auths
  • Introduced noAuthAvailableError for consistent error reporting when no auth is available
• sdk/cliproxy/auth/conductor_overrides_test.go
  • Added tests to verify MarkResult behavior regarding cooldowns when alternative auths are present or absent
• sdk/cliproxy/auth/selector.go
  • Added StickyRoundRobinSelector to provide session-key based routing
  • Updated getAvailableAuths to return a StatusServiceUnavailable error when no auths are available
• sdk/cliproxy/auth/selector_test.go
  • Added test case for SelectorPick_AllUnavailableReturnsServiceUnavailable
• sdk/cliproxy/builder.go
  • Added sticky-round-robin as a selectable routing strategy in the service builder
• sdk/cliproxy/service.go
  • Added sticky-round-robin to the list of normalized routing strategies
  • Implemented backfillAntigravityModels to populate model lists for Antigravity auths using a primary source if their own list is empty
• sdk/cliproxy/service_antigravity_backfill_test.go
  • Added new test file for Antigravity model backfilling functionality, including tests for registering missing auths and respecting excluded models
• test/builtin_tools_translation_test.go
  • Updated import paths to the new repository location
  • Modified TestOpenAIResponsesToOpenAI_PassesBuiltinTools to assert that built-in tools are now passed through in OpenAI Chat Completions translation
• test/openai_websearch_translation_test.go
  • Added new test file to cover translation of OpenAI web search annotations to Claude, Gemini, and OpenAI Responses formats for both streaming and non-streaming scenarios

Ignored Files

• Ignored by pattern: .github/workflows/** (1)
  • .github/workflows/release.yaml

Activity

• The pull request was generated using Claude Code, indicating an automated process for integrating these features.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request integrates a significant number of features from previously unmerged pull requests, enhancing functionality across various components. Key improvements include the introduction of a 'sticky-round-robin' routing strategy, caching for antigravity models to improve resilience, and more robust error handling and retry logic in the iFlow executor. The translation layers have also been updated to handle annotations and improve compatibility. While the changes are substantial and largely beneficial, I've identified a few critical bugs in the openai_claude_response.go translator that will cause compilation errors due to variable re-declarations and type mismatches. These issues need to be addressed to ensure the stability of the new features.