feat: replicate channel flux model#2190
Conversation
WalkthroughAdds support for a new Replicate channel for image generation using Flux models. Introduces API type and channel constants, implements a relay channel adaptor with OpenAI-to-Flux image request conversion, adds special-case response handling for 201 status codes, updates model pricing, and extends frontend UI components. Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant ImageHandler as Image Handler
participant Adaptor as Replicate Adaptor
participant ReplicateAPI as Replicate API
participant Storage as Storage/Download
Client->>ImageHandler: POST /v1/images/generations (OpenAI format)
ImageHandler->>Adaptor: ConvertImageRequest(OpenAI request)
Adaptor->>Adaptor: mapOpenAISizeToFlux(size to aspect ratio)
Adaptor->>Adaptor: uploadFileFromForm(if editing)
Adaptor-->>ImageHandler: Flux-compatible request payload
ImageHandler->>Adaptor: DoRequest(Flux payload)
Adaptor->>ReplicateAPI: POST /predictions (Flux payload)
ReplicateAPI-->>Adaptor: 201 Created + prediction response
Adaptor->>Adaptor: DoResponse(parse Flux output)
note over Adaptor: Extract output URLs from Flux response
Adaptor->>Storage: downloadImagesToBase64(if needed)
Storage-->>Adaptor: base64-encoded images
Adaptor-->>ImageHandler: OpenAI-format response
ImageHandler->>Client: 200 OK with image URLs/data
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes
Possibly related PRs
Suggested reviewers
Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
relay/image_handler.go (1)
96-104: Consider moving 201 status handling to the Replicate adaptor.The special handling for 201 Created responses from Replicate works correctly, but it adds API-specific logic to the shared relay layer. This creates coupling between the generic ImageHelper and Replicate-specific behavior.
Consider handling this in the Replicate adaptor's
DoResponsemethod instead:
- The adaptor could normalize the status code before returning the response
- This would keep the relay layer more generic and maintainable
- Other adaptors with similar quirks wouldn't need relay-layer changes
However, if multiple Replicate endpoints return 201 for success, keeping it here may be more maintainable than duplicating the check across multiple adaptor methods.
If you decide this is the right place for this logic, the current implementation is correct. Alternatively, apply this pattern in the adaptor:
In
relay/channel/replicate/adaptor.goDoResponse method:func (a *Adaptor) DoResponse(c *gin.Context, resp *http.Response, info *relaycommon.RelayInfo) (usage any, err *types.NewAPIError) { // Normalize 201 Created to 200 OK for Prefer: wait responses if resp.StatusCode == http.StatusCreated { resp.StatusCode = http.StatusOK } // ... rest of response handling }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (11)
common/api_type.go(1 hunks)constant/api_type.go(1 hunks)constant/channel.go(3 hunks)relay/channel/replicate/adaptor.go(1 hunks)relay/channel/replicate/constants.go(1 hunks)relay/channel/replicate/dto.go(1 hunks)relay/image_handler.go(2 hunks)relay/relay_adaptor.go(2 hunks)setting/ratio_setting/model_ratio.go(1 hunks)web/src/constants/channel.constants.js(1 hunks)web/src/helpers/render.jsx(2 hunks)
🧰 Additional context used
🧠 Learnings (2)
📚 Learning: 2025-08-05T17:14:17.246Z
Learnt from: neotf
Repo: QuantumNous/new-api PR: 1511
File: setting/ratio_setting/model_ratio.go:118-123
Timestamp: 2025-08-05T17:14:17.246Z
Learning: Claude models handle "-thinking" variants differently from Gemini models. For Claude models, only the base model (without "-thinking") gets an entry in defaultModelRatio map. The "-thinking" variants rely on the Claude relay handler stripping the suffix using strings.TrimSuffix(textRequest.Model, "-thinking") before looking up the ratio, so they automatically use the base model's ratio.
Applied to files:
setting/ratio_setting/model_ratio.go
📚 Learning: 2025-08-08T17:12:43.157Z
Learnt from: RedwindA
Repo: QuantumNous/new-api PR: 1537
File: relay/gemini_handler.go:330-342
Timestamp: 2025-08-08T17:12:43.157Z
Learning: In the new-api repository, the `GeminiEmbeddingHandler` function in `relay/gemini_handler.go` is designed specifically for native Gemini embedding requests and therefore does not require the `ConvertGeminiRequest` step that is used in the chat handler. The embedding requests are already in the native Gemini format and don't need conversion.
Applied to files:
relay/channel/replicate/adaptor.go
🧬 Code graph analysis (4)
common/api_type.go (2)
constant/channel.go (1)
ChannelTypeReplicate(56-56)constant/api_type.go (1)
APITypeReplicate(37-37)
relay/relay_adaptor.go (3)
constant/api_type.go (1)
APITypeReplicate(37-37)relay/channel/replicate/adaptor.go (1)
Adaptor(27-28)relay/channel/adapter.go (1)
Adaptor(15-32)
relay/image_handler.go (2)
constant/api_type.go (1)
APITypeReplicate(37-37)service/error.go (2)
RelayErrorHandler(84-113)ResetStatusCode(115-132)
relay/channel/replicate/adaptor.go (12)
relay/common/relay_info.go (1)
RelayInfo(76-123)constant/channel.go (2)
ChannelBaseURLs(61-119)ChannelTypeReplicate(56-56)relay/common/relay_utils.go (1)
GetFullRequestURL(25-37)relay/channel/api_request.go (3)
SetupApiRequestHeader(27-39)DoRequest(252-254)DoApiRequest(61-90)relay/channel/replicate/constants.go (3)
ModelFlux11Pro(7-7)ModelList(10-12)ChannelName(5-5)common/json.go (2)
Unmarshal(9-11)Marshal(21-23)relay/constant/relay_mode.go (1)
RelayModeImagesEdits(15-15)types/error.go (5)
NewAPIError(87-95)NewError(204-224)ErrorCodeBadResponse(71-71)ErrorCodeReadResponseBodyFailed(69-69)ErrorCodeBadResponseBody(72-72)dto/openai_image.go (1)
ImageData(171-175)dto/openai_response.go (1)
Usage(222-240)service/image.go (1)
GetImageFromUrl(68-116)service/http_client.go (1)
GetHttpClient(49-51)
🔇 Additional comments (7)
constant/channel.go (1)
56-56: LGTM! Replicate channel constant properly integrated.The new ChannelTypeReplicate constant (value 56) is correctly added with its corresponding base URL and name mapping, following the established pattern for other channels.
Also applies to: 118-118, 174-174
common/api_type.go (1)
74-75: LGTM! Channel-to-API type mapping correctly added.The mapping from ChannelTypeReplicate to APITypeReplicate follows the established pattern and is properly integrated into the switch statement.
web/src/constants/channel.constants.js (1)
182-186: LGTM! UI channel option properly configured.The Replicate channel option (value 56) is correctly added with appropriate color and label, consistent with backend constants and other channel entries.
relay/relay_adaptor.go (1)
29-29: LGTM! Adaptor factory properly extended.The replicate adaptor is correctly imported and wired into the GetAdaptor factory function. The import maintains alphabetical order, and the switch case follows the established pattern.
Note: The actual replicate adaptor implementation in
relay/channel/replicate/adaptor.goshould be reviewed separately to ensure it properly implements the Adaptor interface.Also applies to: 117-118
constant/api_type.go (1)
37-37: LGTM! API type constant correctly positioned.APITypeReplicate is properly added to the iota sequence, positioned before the sentinel APITypeDummy constant, following the established pattern.
web/src/helpers/render.jsx (1)
58-58: LGTM! Icon rendering properly configured.The Replicate icon is correctly imported from @lobehub/icons and wired into the getChannelIcon function for channel type 56, following the established pattern for other channel icons.
Also applies to: 346-347
setting/ratio_setting/model_ratio.go (1)
271-297: Changes verified: intentional model replacement with no price modifications.Verification confirms the diff shows an intentional swap: "imagen-3.0-generate-002" ($0.03) was removed and replaced with "black-forest-labs/flux-1.1-pro" ($0.04). No existing model prices were modified—only this one substitution and reordering. The remaining entries maintain their original prices, so this change is safe and intentional.
…annel feat: replicate channel flux model
2 participants
support replica channels ( https://replicate.com/ ) image generation model
support using
/v1/images/edits&/v1/images/generationsreq format
{ "model": "black-forest-labs/flux-1.1-pro", "prompt": "A cute baby sea otter" }response format
{ "data": [ { "url": "https://replicate.delivery/path/to/the/generated/image.webp", "b64_json": "", "revised_prompt": "" } ], "created": 1762510317 }Summary by CodeRabbit