feat: Get live provider and model data from model.dev#11007
feat: Get live provider and model data from model.dev#11007deon-sanchez wants to merge 47 commits into
Conversation
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the WalkthroughThis PR adds live model data fetching from the models.dev API with caching support, controlled by the Changes
Sequence Diagram(s)sequenceDiagram
participant Client as Client Code
participant UM as unified_models
participant MDC as models_dev_client
participant Cache as Local Cache File
participant API as models.dev API
Client->>UM: get_models_detailed()
alt USE_LIVE_MODEL_DATA enabled
UM->>MDC: get_live_models_detailed()
MDC->>Cache: _load_cache()
alt Cache valid (within 1h TTL)
Cache-->>MDC: cached data
else Cache missing or expired
MDC->>API: fetch https://models.dev/api.json
API-->>MDC: provider & model data
MDC->>Cache: _save_cache(data)
end
loop For each provider & model
MDC->>MDC: transform_api_model_to_metadata()
Note over MDC: Map provider IDs,<br/>determine model type,<br/>structure cost/limits
end
MDC-->>UM: list[ModelMetadata]
else Live data unavailable
UM->>UM: get_static_models_detailed()
Note over UM: Fallback to static<br/>model definitions
end
UM-->>Client: list[ModelMetadata]
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes
Possibly related PRs
Suggested labels
Suggested reviewers
Pre-merge checks and finishing touchesImportant Pre-merge checks failedPlease resolve all errors before merging. Addressing warnings is optional. ❌ Failed checks (1 error, 1 warning, 1 inconclusive)
✅ Passed checks (4 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Frontend Unit Test Coverage ReportCoverage Summary
Unit Test Results
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #11007 +/- ##
==========================================
- Coverage 33.23% 32.77% -0.46%
==========================================
Files 1394 1396 +2
Lines 66068 66504 +436
Branches 9778 9883 +105
==========================================
- Hits 21956 21795 -161
- Misses 42986 43563 +577
- Partials 1126 1146 +20
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (4)
src/lfx/src/lfx/base/models/unified_models.py (2)
100-113: Consider narrowing the exception type (static analysis warning).The blind
Exceptioncatch at line 109 triggers Ruff BLE001. While the fallback behavior is correct for resilience, consider either:
- Catching more specific exceptions (e.g.,
OSError,ValueError,TimeoutError)- Adding
# noqa: BLE001with a comment explaining why broad catching is intentional here- except Exception as e: + except Exception as e: # noqa: BLE001 - Intentional broad catch for graceful fallback logger.debug(f"Failed to get live provider metadata: {e}")
133-155: Same BLE001 warning applies here.Line 153 has the same blind exception catch. Add
# noqa: BLE001with justification for consistency.- except Exception as e: + except Exception as e: # noqa: BLE001 - Intentional broad catch for graceful fallback logger.debug(f"Failed to fetch live models: {e}") return []src/lfx/src/lfx/base/models/models_dev_client.py (2)
174-189: Use idiomatic string containment check.Line 186 uses
.find() != -1which is not idiomatic Python. Use theinoperator instead.Apply this diff:
- if model_data.get("id", "").lower().find("embed") != -1: + if "embed" in model_data.get("id", "").lower():
191-227: Consider omitting None values from TypedDict construction.The transform functions explicitly include optional fields with
Nonevalues (e.g., lines 199-203). Since these TypedDicts havetotal=False, it's cleaner to omitNonevalues rather than include them, which also prevents potential serialization issues.Example for
_transform_cost:def _transform_cost(cost_data: dict[str, Any] | None) -> ModelCost | None: """Transform API cost data to ModelCost format.""" if not cost_data: return None - return ModelCost( - input=cost_data.get("input", 0), - output=cost_data.get("output", 0), - reasoning=cost_data.get("reasoning"), - cache_read=cost_data.get("cache_read"), - cache_write=cost_data.get("cache_write"), - input_audio=cost_data.get("input_audio"), - output_audio=cost_data.get("output_audio"), - ) + result = ModelCost( + input=cost_data.get("input", 0), + output=cost_data.get("output", 0), + ) + # Only add optional fields if present + for key in ["reasoning", "cache_read", "cache_write", "input_audio", "output_audio"]: + if key in cost_data: + result[key] = cost_data[key] + return resultApply similar logic to
_transform_limitsand_transform_modalities.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (7)
.env.example(2 hunks)src/frontend/src/controllers/API/queries/models/use-get-model-providers.ts(1 hunks)src/frontend/src/modals/modelProviderModal/index.tsx(1 hunks)src/lfx/src/lfx/base/models/__init__.py(1 hunks)src/lfx/src/lfx/base/models/model_metadata.py(1 hunks)src/lfx/src/lfx/base/models/models_dev_client.py(1 hunks)src/lfx/src/lfx/base/models/unified_models.py(5 hunks)
🧰 Additional context used
📓 Path-based instructions (3)
src/frontend/src/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/frontend_development.mdc)
src/frontend/src/**/*.{ts,tsx}: Use React 18 with TypeScript for frontend development
Use Zustand for state management
Files:
src/frontend/src/modals/modelProviderModal/index.tsxsrc/frontend/src/controllers/API/queries/models/use-get-model-providers.ts
src/frontend/src/**/*.{tsx,jsx,css,scss}
📄 CodeRabbit inference engine (.cursor/rules/frontend_development.mdc)
Use Tailwind CSS for styling
Files:
src/frontend/src/modals/modelProviderModal/index.tsx
src/frontend/src/**/*.{tsx,jsx}
📄 CodeRabbit inference engine (.cursor/rules/frontend_development.mdc)
src/frontend/src/**/*.{tsx,jsx}: Implement dark mode support using the useDarkMode hook and dark store
Use Lucide React for icon components in the application
Files:
src/frontend/src/modals/modelProviderModal/index.tsx
🧠 Learnings (5)
📚 Learning: 2025-07-11T22:12:46.255Z
Learnt from: namastex888
Repo: langflow-ai/langflow PR: 9018
File: src/frontend/src/modals/apiModal/codeTabs/code-tabs.tsx:244-244
Timestamp: 2025-07-11T22:12:46.255Z
Learning: In src/frontend/src/modals/apiModal/codeTabs/code-tabs.tsx, the inconsistent showLineNumbers setting between Step 1 (false) and Step 2 (true) in the API modal is intentional to prevent breaking the modal height. Step 1 uses showLineNumbers={false} to save vertical space while Step 2 uses showLineNumbers={true} for better readability of longer code.
Applied to files:
src/frontend/src/modals/modelProviderModal/index.tsx
📚 Learning: 2025-07-23T21:19:22.567Z
Learnt from: deon-sanchez
Repo: langflow-ai/langflow PR: 9158
File: src/backend/base/langflow/api/v1/mcp_projects.py:404-404
Timestamp: 2025-07-23T21:19:22.567Z
Learning: In langflow MCP projects configuration, prefer using dynamically computed URLs (like the `sse_url` variable) over hardcoded localhost URLs to ensure compatibility across different deployment environments.
Applied to files:
.env.example
📚 Learning: 2025-11-24T19:46:57.920Z
Learnt from: CR
Repo: langflow-ai/langflow PR: 0
File: .cursor/rules/icons.mdc:0-0
Timestamp: 2025-11-24T19:46:57.920Z
Learning: Applies to src/frontend/src/icons/lazyIconImports.ts : Add icon entries to the `lazyIconsMapping` object in `src/frontend/src/icons/lazyIconImports.ts`. The key must match the backend icon name exactly (case-sensitive) and use dynamic imports: `IconName: () => import("@/icons/IconName").then((mod) => ({ default: mod.IconNameIcon }))`.
Applied to files:
src/frontend/src/controllers/API/queries/models/use-get-model-providers.ts
📚 Learning: 2025-11-24T19:46:57.920Z
Learnt from: CR
Repo: langflow-ai/langflow PR: 0
File: .cursor/rules/icons.mdc:0-0
Timestamp: 2025-11-24T19:46:57.920Z
Learning: Use clear, recognizable icon names (e.g., `"AstraDB"`, `"Postgres"`, `"OpenAI"`). Always use the same icon name for the same service across backend and frontend.
Applied to files:
src/frontend/src/controllers/API/queries/models/use-get-model-providers.ts
📚 Learning: 2025-06-23T12:46:52.420Z
Learnt from: CR
Repo: langflow-ai/langflow PR: 0
File: .cursor/rules/icons.mdc:0-0
Timestamp: 2025-06-23T12:46:52.420Z
Learning: The frontend icon mapping key (in 'lazyIconsMapping') must match the backend 'icon' attribute string exactly, including case sensitivity, to ensure correct icon rendering.
Applied to files:
src/frontend/src/controllers/API/queries/models/use-get-model-providers.ts
🧬 Code graph analysis (2)
src/lfx/src/lfx/base/models/models_dev_client.py (1)
src/lfx/src/lfx/base/models/model_metadata.py (4)
ModelCost(4-13)ModelLimits(16-20)ModelMetadata(30-69)ModelModalities(23-27)
src/lfx/src/lfx/base/models/unified_models.py (1)
src/lfx/src/lfx/base/models/models_dev_client.py (3)
clear_cache(386-394)get_live_models_detailed(307-350)get_provider_metadata_from_api(354-383)
🪛 GitHub Actions: Ruff Style Check
src/lfx/src/lfx/base/models/__init__.py
[error] 20-20: RUF022 __all__ is not sorted.
🪛 GitHub Check: Ruff Style Check (3.13)
src/lfx/src/lfx/base/models/models_dev_client.py
[failure] 169-169: Ruff (BLE001)
src/lfx/src/lfx/base/models/models_dev_client.py:169:12: BLE001 Do not catch blind exception: Exception
[failure] 159-159: Ruff (TRY300)
src/lfx/src/lfx/base/models/models_dev_client.py:159:9: TRY300 Consider moving this statement to an else block
src/lfx/src/lfx/base/models/unified_models.py
[failure] 109-109: Ruff (BLE001)
src/lfx/src/lfx/base/models/unified_models.py:109:12: BLE001 Do not catch blind exception: Exception
[failure] 189-189: Ruff (PLW0602)
src/lfx/src/lfx/base/models/unified_models.py:189:12: PLW0602 Using global for MODELS_DETAILED but no assignment is done
[failure] 153-153: Ruff (BLE001)
src/lfx/src/lfx/base/models/unified_models.py:153:12: BLE001 Do not catch blind exception: Exception
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
- GitHub Check: Test Docker Images / Test docker images
- GitHub Check: Run Frontend Tests / Determine Test Suites and Shard Distribution
- GitHub Check: Run Frontend Unit Tests / Frontend Jest Unit Tests
- GitHub Check: Test Starter Templates
- GitHub Check: Update Component Index
- GitHub Check: Run Ruff Check and Format
- GitHub Check: Update Starter Projects
🔇 Additional comments (13)
src/frontend/src/modals/modelProviderModal/index.tsx (1)
255-255: LGTM! Enables scrolling for overflowing provider list.The addition of
overflow-y-autocorrectly handles scenarios where the provider list exceeds the fixed height of 513px, allowing users to scroll through all available providers..env.example (1)
143-148: Documentation looks good.The environment variable documentation is clear and follows the established pattern in this file. It properly documents the purpose, allowed values, default, and fallback behavior.
src/lfx/src/lfx/base/models/model_metadata.py (2)
4-28: Well-structured type definitions.The new
ModelCost,ModelLimits, andModelModalitiesTypedDicts are well-designed withtotal=Falsefor optional fields and clear inline documentation of units and purposes.
72-144: Clean implementation of the factory function.The conditional assignment pattern (only adding fields when not
None) is a good approach to keep the metadata dictionaries lean. The function signature is getting long but remains manageable for a configuration factory.src/lfx/src/lfx/base/models/unified_models.py (3)
33-35: Environment variable parsing looks good.The pattern of parsing the env var with a lowercase comparison and default to
"false"is correct and defensive.
158-175: Good fallback pattern with appropriate logging.The live-to-static fallback with a warning log is a solid resilience pattern. The function cleanly encapsulates the toggle behavior.
181-200: > Likely an incorrect or invalid review comment.src/frontend/src/controllers/API/queries/models/use-get-model-providers.ts (1)
78-100: Verify icon keys matchlazyIconImports.tsbefore merging.The provider-to-icon mappings reference several custom icons (GoogleGenerativeAI, VertexAI, Mistral, DeepSeek, xAI, WatsonxAI, SambaNova, AWS, Azure, NVIDIA, Ollama) that must be registered in
src/frontend/src/icons/lazyIconImports.ts. Icon keys are case-sensitive and must match exactly. Ensure each referenced icon key has a corresponding dynamic import entry inlazyIconImports.ts. The "Bot" fallback is acceptable for unsupported providers (Together AI, Fireworks AI, Alibaba, Cerebras).src/lfx/src/lfx/base/models/models_dev_client.py (5)
1-93: LGTM: Well-structured module with comprehensive provider mappings.The imports, constants, and provider mappings are well-organized and comprehensive.
96-120: LGTM: Cache loading logic is sound.The cache path construction and loading with TTL validation are implemented correctly.
122-131: LGTM: Cache saving is implemented correctly.Directory creation and error handling are appropriate.
307-351: LGTM: Query function is well-implemented.The filtering logic and parameter handling are correct.
397-439: LGTM: Convenience and search functions are well-implemented.The wrapper and search functions provide a clean API surface.
github-actions
Bot
added
enhancement
github-actions
Bot
added
enhancement
github-actions
Bot
added
enhancement
github-actions
Bot
added
enhancement
github-actions
Bot
added
enhancement
github-actions
Bot
added
enhancement
github-actions
Bot
added
enhancement
github-actions
Bot
added
enhancement
github-actions
Bot
added
enhancement
…or output limit - Added MAX_EMBEDDING_OUTPUT_LIMIT constant (4096) for embedding dimension threshold - Simplified _is_embedding_model return statement by combining conditions - Improved code readability with named constant instead of magic number
github-actions
Bot
added
enhancement
github-actions
Bot
added
enhancement
github-actions
Bot
added
enhancement
github-actions
Bot
added
enhancement
github-actions
Bot
added
enhancement
github-actions
Bot
added
enhancement
1 participant
This pull request introduces improvements to model selection flexibility and configuration, as well as an environment variable for enabling live model data fetching. The most significant changes include allowing multiple model types to be filtered in the API, updating starter project components to specify model type, and adding a new environment variable for live model data.
API and Model Selection Enhancements:
list_modelsAPI endpoint now accepts multiple model types (such asllms,embeddings,audio,video) via a repeated query parameter, allowing for more flexible model filtering.Basic Prompt Chaining.jsonstarter project, theModelInputfor the language model component now explicitly setsmodel_type=["llm"], ensuring that only language models are selectable in the UI. [1] [2] [3]Configuration and Environment:
LFX_USE_LIVE_MODEL_DATA, is added to.env.exampleto optionally enable fetching live model data from the models.dev API instead of using static constants. This provides more up-to-date model options if enabled.