feat: OpenAI compatibility#9069
Conversation
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the WalkthroughThis change introduces an OpenAI-compatible responses API endpoint to the backend, including new request/response schemas, endpoint logic for streaming and non-streaming responses, and integration tests. The router is registered with the API, and relevant exports are updated. The tests validate both streaming and non-streaming OpenAI-compatible interactions. Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant API
participant FlowEngine
participant Telemetry
Client->>API: POST /v1/responses (OpenAI request)
API->>FlowEngine: Validate & run flow (by flow_id)
alt Streaming enabled
FlowEngine-->>API: Stream response chunks
API-->>Client: Stream SSE events (OpenAI format)
else Non-streaming
FlowEngine-->>API: Final response
API-->>Client: OpenAI-compatible response
end
API-->>Telemetry: Log usage and events (async)
Suggested labels
Suggested reviewers
✨ Finishing Touches🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
Status, Documentation and Community
|
dosubot
Bot
added
the
size:XL
github-actions
Bot
added
enhancement
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (1)
src/backend/base/langflow/api/v1/openai_responses.py (1)
43-48: Improve function signature with explicit keyword arguments.The function has boolean positional arguments which can be confusing. Consider making the
streamparameter keyword-only.-async def run_flow_for_openai_responses( - flow: FlowRead, - request: OpenAIResponsesRequest, - api_key_user: UserRead, - stream: bool = False, -) -> OpenAIResponsesResponse | StreamingResponse: +async def run_flow_for_openai_responses( + flow: FlowRead, + request: OpenAIResponsesRequest, + api_key_user: UserRead, + *, + stream: bool = False, +) -> OpenAIResponsesResponse | StreamingResponse:
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (5)
src/backend/base/langflow/api/router.py(2 hunks)src/backend/base/langflow/api/v1/__init__.py(2 hunks)src/backend/base/langflow/api/v1/openai_responses.py(1 hunks)src/backend/base/langflow/schema/openai_responses_schemas.py(1 hunks)src/backend/tests/integration/test_openai_responses_integration.py(1 hunks)
🧰 Additional context used
📓 Path-based instructions (4)
src/backend/**/*.py
Instructions used from:
Sources:
📄 CodeRabbit Inference Engine
- .cursor/rules/backend_development.mdc
src/backend/tests/**/*.py
Instructions used from:
Sources:
📄 CodeRabbit Inference Engine
- .cursor/rules/testing.mdc
{src/backend/tests/**/*.py,src/frontend/**/*.test.{ts,tsx,js,jsx},src/frontend/**/*.spec.{ts,tsx,js,jsx},tests/**/*.py}
Instructions used from:
Sources:
📄 CodeRabbit Inference Engine
- .cursor/rules/testing.mdc
{src/backend/tests/**/*.py,tests/**/*.py}
Instructions used from:
Sources:
📄 CodeRabbit Inference Engine
- .cursor/rules/testing.mdc
🧠 Learnings (4)
src/backend/base/langflow/api/v1/__init__.py (1)
Learnt from: CR
PR: langflow-ai/langflow#0
File: .cursor/rules/backend_development.mdc:0-0
Timestamp: 2025-06-30T14:39:17.464Z
Learning: Applies to src/backend/base/langflow/components/**/__init__.py : Update __init__.py with alphabetical imports when adding new components
src/backend/base/langflow/api/v1/openai_responses.py (2)
Learnt from: CR
PR: langflow-ai/langflow#0
File: .cursor/rules/testing.mdc:0-0
Timestamp: 2025-06-30T14:41:58.849Z
Learning: Applies to {src/backend/tests/**/*.py,tests/**/*.py} : Test Langflow's REST API endpoints using the async 'client' fixture and assert correct status codes and response structure.
Learnt from: CR
PR: langflow-ai/langflow#0
File: .cursor/rules/testing.mdc:0-0
Timestamp: 2025-06-30T14:41:58.849Z
Learning: Applies to {src/backend/tests/**/*.py,tests/**/*.py} : Use predefined JSON flows and utility functions for flow testing (e.g., 'create_flow', 'build_flow', 'get_build_events', 'consume_and_assert_stream').
src/backend/tests/integration/test_openai_responses_integration.py (11)
Learnt from: CR
PR: langflow-ai/langflow#0
File: .cursor/rules/testing.mdc:0-0
Timestamp: 2025-06-30T14:41:58.849Z
Learning: Applies to {src/backend/tests/**/*.py,tests/**/*.py} : Test Langflow's REST API endpoints using the async 'client' fixture and assert correct status codes and response structure.
Learnt from: CR
PR: langflow-ai/langflow#0
File: .cursor/rules/testing.mdc:0-0
Timestamp: 2025-06-30T14:41:58.849Z
Learning: Applies to {src/backend/tests/**/*.py,tests/**/*.py} : Use 'anyio' and 'aiofiles' for async file operations in tests.
Learnt from: CR
PR: langflow-ai/langflow#0
File: .cursor/rules/testing.mdc:0-0
Timestamp: 2025-06-30T14:41:58.849Z
Learning: Applies to src/backend/tests/**/*.py : Use the 'client' fixture (an async httpx.AsyncClient) for API tests, as defined in 'src/backend/tests/conftest.py'.
Learnt from: CR
PR: langflow-ai/langflow#0
File: .cursor/rules/backend_development.mdc:0-0
Timestamp: 2025-06-30T14:39:17.464Z
Learning: Applies to src/backend/tests/unit/**/*.py : Test component integration within flows using create_flow, build_flow, and get_build_events utilities
Learnt from: CR
PR: langflow-ai/langflow#0
File: .cursor/rules/testing.mdc:0-0
Timestamp: 2025-06-30T14:41:58.849Z
Learning: Applies to {src/backend/tests/**/*.py,tests/**/*.py} : Use predefined JSON flows and utility functions for flow testing (e.g., 'create_flow', 'build_flow', 'get_build_events', 'consume_and_assert_stream').
Learnt from: CR
PR: langflow-ai/langflow#0
File: .cursor/rules/testing.mdc:0-0
Timestamp: 2025-06-30T14:41:58.849Z
Learning: Applies to {src/backend/tests/**/*.py,tests/**/*.py} : Be aware of ContextVar propagation in async tests and test both direct event loop execution and 'asyncio.to_thread' scenarios.
Learnt from: CR
PR: langflow-ai/langflow#0
File: .cursor/rules/backend_development.mdc:0-0
Timestamp: 2025-06-30T14:39:17.464Z
Learning: Applies to src/backend/tests/unit/**/*.py : Use pytest.mark.api_key_required and pytest.mark.no_blockbuster for tests involving external APIs
Learnt from: CR
PR: langflow-ai/langflow#0
File: .cursor/rules/testing.mdc:0-0
Timestamp: 2025-06-30T14:41:58.849Z
Learning: Applies to {src/backend/tests/**/*.py,tests/**/*.py} : Use '@pytest.mark.asyncio' for async test functions.
Learnt from: CR
PR: langflow-ai/langflow#0
File: .cursor/rules/testing.mdc:0-0
Timestamp: 2025-06-30T14:41:58.849Z
Learning: Applies to {src/backend/tests/**/*.py,src/frontend/**/*.test.{ts,tsx,js,jsx},src/frontend/**/*.spec.{ts,tsx,js,jsx},tests/**/*.py} : Test both sync and async code paths in components.
Learnt from: CR
PR: langflow-ai/langflow#0
File: .cursor/rules/testing.mdc:0-0
Timestamp: 2025-06-30T14:41:58.849Z
Learning: Applies to {src/backend/tests/**/*.py,src/frontend/**/*.test.{ts,tsx,js,jsx},src/frontend/**/*.spec.{ts,tsx,js,jsx},tests/**/*.py} : Mock external dependencies appropriately in tests.
Learnt from: CR
PR: langflow-ai/langflow#0
File: .cursor/rules/testing.mdc:0-0
Timestamp: 2025-06-30T14:41:58.849Z
Learning: Applies to {src/backend/tests/**/*.py,tests/**/*.py} : Test Langflow's 'Message' objects and chat functionality by asserting correct properties and structure.
src/backend/base/langflow/schema/openai_responses_schemas.py (1)
Learnt from: CR
PR: langflow-ai/langflow#0
File: .cursor/rules/backend_development.mdc:0-0
Timestamp: 2025-06-30T14:39:17.464Z
Learning: Applies to src/backend/base/langflow/services/database/models/**/*.py : Place database models in src/backend/base/langflow/services/database/models/ and its subdirectories
🪛 GitHub Check: Ruff Style Check (3.13)
src/backend/base/langflow/api/v1/openai_responses.py
[failure] 359-359: Ruff (TRY300)
src/backend/base/langflow/api/v1/openai_responses.py:359:9: TRY300 Consider moving this statement to an else block
[failure] 215-215: Ruff (BLE001)
src/backend/base/langflow/api/v1/openai_responses.py:215:20: BLE001 Do not catch blind exception: Exception
[failure] 181-181: Ruff (E501)
src/backend/base/langflow/api/v1/openai_responses.py:181:121: E501 Line too long (142 > 120)
[failure] 165-165: Ruff (E501)
src/backend/base/langflow/api/v1/openai_responses.py:165:121: E501 Line too long (139 > 120)
[failure] 138-138: Ruff (E501)
src/backend/base/langflow/api/v1/openai_responses.py:138:121: E501 Line too long (124 > 120)
[failure] 52-52: Ruff (EM101)
src/backend/base/langflow/api/v1/openai_responses.py:52:26: EM101 Exception must not use a string literal, assign to variable first
[failure] 52-52: Ruff (TRY003)
src/backend/base/langflow/api/v1/openai_responses.py:52:15: TRY003 Avoid specifying long messages outside the exception class
[failure] 47-47: Ruff (FBT002)
src/backend/base/langflow/api/v1/openai_responses.py:47:5: FBT002 Boolean default positional argument in function definition
[failure] 47-47: Ruff (FBT001)
src/backend/base/langflow/api/v1/openai_responses.py:47:5: FBT001 Boolean-typed positional argument in function definition
[failure] 37-40: Ruff (SIM110)
src/backend/base/langflow/api/v1/openai_responses.py:37:5: SIM110 Use return any(node.get("data", {}).get("type") in ["ChatInput", "Chat Input"] for node in flow_data["nodes"]) instead of for loop
🪛 GitHub Actions: Ruff Style Check
src/backend/base/langflow/api/v1/openai_responses.py
[error] 37-37: SIM110: Use return any(node.get("data", {}).get("type") in ["ChatInput", "Chat Input"] for node in flow_data["nodes"]) instead of for loop.
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Optimize new Python code in this PR
🔇 Additional comments (17)
src/backend/base/langflow/api/router.py (2)
15-15: LGTM: Import follows existing pattern.The import of
openai_responses_routeris correctly added alongside other router imports.
52-52: LGTM: Router inclusion follows existing pattern.The
openai_responses_routeris correctly included in the v1 router following the same pattern as other routers.src/backend/base/langflow/api/v1/__init__.py (2)
11-11: LGTM: Import added in correct alphabetical order.The import of
openai_responses_routeris correctly placed in alphabetical order among other router imports.
31-31: LGTM: Export added in correct alphabetical order.The
openai_responses_routeris correctly added to the__all__list in alphabetical order.src/backend/base/langflow/schema/openai_responses_schemas.py (5)
6-17: LGTM: Well-structured request model.The
OpenAIResponsesRequestmodel is well-defined with appropriate field descriptions and default values. The use ofmodelfor flow_id is a clever way to maintain OpenAI compatibility.
19-45: LGTM: Comprehensive response model.The
OpenAIResponsesResponsemodel covers all necessary fields for OpenAI compatibility with sensible defaults. The use ofField(default_factory=...)for complex default values is appropriate.
47-56: LGTM: Streaming chunk model is well-defined.The
OpenAIResponsesStreamChunkmodel properly represents streaming response chunks with the correct object type and required fields.
58-60: LGTM: Simple error response model.The
OpenAIErrorResponsemodel is appropriately minimal for error responses.
62-71: LGTM: Useful error creation utility.The
create_openai_errorfunction provides a consistent way to create OpenAI-compatible error responses with optional code parameter.src/backend/base/langflow/api/v1/openai_responses.py (3)
77-287: LGTM: Complex streaming logic is well-implemented.The streaming logic handles OpenAI-compatible event generation, tool calls, and proper error handling. The asynchronous generator pattern is correctly implemented with proper cleanup.
289-380: LGTM: Comprehensive endpoint implementation.The endpoint properly handles validation, authentication, flow execution, and error responses. The telemetry logging and OpenAI-compatible error handling are well-implemented.
359-359: Move success return to else block.The static analysis correctly suggests moving the success return statement to an else block for better structure.
- # Log telemetry for successful completion - if not request.stream: # Only log for non-streaming responses - end_time = time.perf_counter() - background_tasks.add_task( - telemetry_service.log_package_run, - RunPayload( - run_is_webhook=False, - run_seconds=int(end_time - start_time), - run_success=True, - run_error_message="", - ), - ) - - return result + # Log telemetry for successful completion + if not request.stream: # Only log for non-streaming responses + end_time = time.perf_counter() + background_tasks.add_task( + telemetry_service.log_package_run, + RunPayload( + run_is_webhook=False, + run_seconds=int(end_time - start_time), + run_success=True, + run_error_message="", + ), + ) + + return resultActually, looking at the context more carefully, this suggestion doesn't seem applicable here since there's no corresponding if/else structure. The static analysis might be incorrectly flagging this.
src/backend/tests/integration/test_openai_responses_integration.py (5)
11-33: LGTM: Well-structured environment variable loading.The environment variable loading logic is robust with multiple fallback paths and appropriate error handling.
35-46: LGTM: Utility function for global variable creation.The
create_global_variablefunction is well-implemented with proper error handling and logging.
48-107: LGTM: Comprehensive flow preparation logic.The
load_and_prepare_flowfunction handles all necessary setup steps including global variable creation, flow loading, and build completion polling. The timeout handling is appropriate.
109-142: LGTM: Thorough non-streaming test.The test properly validates the non-streaming OpenAI responses endpoint with comprehensive error handling and response validation.
144-184: LGTM: Comprehensive streaming test.The test properly validates the streaming OpenAI responses endpoint, checking for server-sent events structure and the required
[DONE]marker. The event parsing and validation logic is well-implemented.
github-actions
Bot
added
enhancement
…patibility`)
Here’s an optimized rewrite of your function. The two main bottlenecks are.
1. The repeated construction of `{"data":{}}` default dictionaries,
2. The linear-time list search for `in ["ChatInput", "Chat Input"]` every iteration,
3. Unnecessary iteration after the match is found.
**Optimizations:**
- Make the lookup a set for O(1) searches: `{"ChatInput", "Chat Input"}`
- Use explicit checks instead of chained `.get()` to avoid unnecessary dict allocations
- Use early `return` as before.
### Why it's faster.
- **Set lookup:** O(1) membership test instead of O(N)
- **Fewer intermediate dicts:** No unnecessary `get("data", {})` allocations per iteration
- **Variable caching:** Keeps `chat_types` as a variable (set) instead of creating a list each loop.
This change preserves return values and functionality. If your data is especially large or frequently queried, this small optimization can give noticeable improvements.
![codeflash-ai[bot]](https://avatars.githubusercontent.com/in/413177?s=60&v=4){kind=small}
…ity and consistency. Replace print statements with appropriate logging levels, enhancing error handling and debugging capabilities.
…ility. Enhance error messaging in `run_flow_for_openai_responses` for clarity. Update response yielding format for better readability. Add noqa comments for linting compliance.
github-actions
Bot
added
enhancement
… handling Extend integration test coverage with new test file containing validation for empty inputs, invalid models, tools parameter rejection, timeout scenarios, and concurrent request handling. Update existing integration tests to improve error handling and response validation.
github-actions
Bot
added
enhancement
| import yaml | ||
| from cachetools import TTLCache | ||
| from langchain_core.documents import Document | ||
| from loguru import logger |
There was a problem hiding this comment.
@ogabrielluiz @carlosrcoelho just a callout to move the logging to struct log fully later in follow up PRs.
fix: specify type for tool_calls in openai_responses.py Updated the type annotation for the tool_calls variable to explicitly define it as a list of dictionaries with string keys and Any values, enhancing type safety and clarity in the code. Co-authored-by: Sebastián Estévez <estevezsebastian@gmail.com>
Codecov Report❌ Patch coverage is ❌ Your patch status has failed because the patch coverage (28.62%) is below the target coverage (40.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## main #9069 +/- ##
==========================================
- Coverage 34.65% 34.63% -0.02%
==========================================
Files 1203 1205 +2
Lines 56717 56931 +214
Branches 5352 5341 -11
==========================================
+ Hits 19655 19719 +64
- Misses 36925 37075 +150
Partials 137 137
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
⚡️ Codeflash found optimizations for this PR📄 67% (0.67x) speedup for
|
|
@phact am getting few errors when I am testing it on locally. The code looks good though. |
|
Testing again to see if the issue was repeatable. |
|
7 participants
Adds openai responses API to langflow.
To use:
You can also pass the sesison_id as
previous_response_idYou can now pass:
to get back the tool call outputs
You can also now override langflow global variables by passing http headers in your openai call like so:
Changes:
This pull request introduces an OpenAI-compatible API for Langflow, enabling seamless integration with OpenAI-style requests and responses. The changes include adding a new router for OpenAI responses, implementing the corresponding endpoints, and defining schemas for OpenAI-compatible requests and responses. Below is a breakdown of the most important changes:
API Enhancements:
openai_responses_routerto the existing routers insrc/backend/base/langflow/api/router.pyandsrc/backend/base/langflow/api/v1/__init__.py. This enables routing for OpenAI-compatible endpoints. [1] [2] [3] [4]Endpoint Implementation:
src/backend/base/langflow/api/v1/openai_responses.py, which defines the/responsesendpoint. This endpoint processes OpenAI-style requests, supports streaming responses, and handles error scenarios with OpenAI-compatible error messages.Schema Definitions:
src/backend/base/langflow/schema/openai_responses_schemas.py, which includes models for OpenAI requests (OpenAIResponsesRequest), responses (OpenAIResponsesResponse), stream chunks (OpenAIResponsesStreamChunk), and error responses (OpenAIErrorResponse). These schemas ensure compatibility with OpenAI's API structure.