fix: multi embedding available models issue and improved logging#10846
Merged
Conversation
Introduces a 'fail_safe_mode' option to the EmbeddingModel component, allowing errors to be logged and None returned instead of raising exceptions. Updates the build_embeddings and update_build_config logic to support this mode for OpenAI, Ollama, and IBM watsonx.ai providers.
…/langflow into fail-safe-embeddings
Contributor
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the WalkthroughThe PR introduces fail-safe error handling to the EmbeddingModelComponent through a new fail_safe_mode boolean input that gracefully handles provider initialization failures across OpenAI, Ollama, and IBM watsonx.ai. Additionally, comprehensive logging instrumentation is added to the OpenSearch multimodal component for improved observability during search and ingestion operations. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes
Possibly related PRs
Suggested labels
Suggested reviewers
Pre-merge checks and finishing touches❌ Failed checks (1 error, 2 warnings)
✅ Passed checks (4 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
Frontend Unit Test Coverage ReportCoverage Summary
Unit Test Results
|
Codecov Report✅ All modified and coverable lines are covered by tests. ❌ Your project status has failed because the head coverage (39.99%) is below the target coverage (60.00%). You can increase the head coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## main #10846 +/- ##
=======================================
Coverage 32.51% 32.51%
=======================================
Files 1370 1370
Lines 63494 63494
Branches 9391 9391
=======================================
+ Hits 20643 20647 +4
+ Misses 41812 41808 -4
Partials 1039 1039
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (4)
src/lfx/src/lfx/components/elastic/opensearch_multimodal.py (3)
52-52: Consider reducing log verbosity in utility function.This INFO-level log in
get_embedding_field_namewill fire on every call, which could be frequent during search operations with multiple models. Since the function returns the field name and it's already logged in context where it matters (lines 817, 965), this log entry adds redundancy.Consider changing to DEBUG level or removing this log:
- logger.info(f"chunk_embedding_{normalize_model_name(model_name)}") + logger.debug(f"chunk_embedding_{normalize_model_name(model_name)}")Or simply remove it since the field name is logged in more meaningful contexts elsewhere.
1324-1379: Consider consolidating verbose model matching logs.The detailed model matching instrumentation is excellent for debugging, but the high number of
self.logcalls (visible to end users) could be overwhelming when processing many models. Lines 1324-1378 generate at least 2-3 logs per operation plus additional logs per model.Consider consolidating related logs into single entries:
- self.log(f"[SEARCH] Models detected in index: {available_models}") - self.log(f"[SEARCH] Available embedding identifiers: {list(embedding_by_model.keys())}") + self.log(f"[SEARCH] Models detected: {available_models}, Available identifiers: {list(embedding_by_model.keys())[:10]}{'...' if len(embedding_by_model) > 10 else ''}")Also consider truncating the identifier list in line 1367 if it becomes very long:
- f"This model will be skipped. Available identifiers: {list(embedding_by_model.keys())}" + f"This model will be skipped. Available identifiers: {list(embedding_by_model.keys())[:10]}{'...' if len(embedding_by_model) > 10 else ''}"
1534-1540: Consider consolidating execution logs.Lines 1534-1540 create 4 separate log entries in quick succession. While the information is valuable, this could be overwhelming in production environments with high query volume.
Consider consolidating into 1-2 log entries:
- self.log(f"[EXEC] Executing search with {len(knn_queries_with_candidates)} KNN queries, limit={limit}") - self.log(f"[EXEC] Embedding models used: {list(query_embeddings.keys())}") - self.log(f"[EXEC] KNN fields being queried: {embedding_fields}") + self.log( + f"[EXEC] Search: {len(knn_queries_with_candidates)} KNN queries, " + f"models={list(query_embeddings.keys())}, fields={embedding_fields}, limit={limit}" + )src/backend/base/langflow/initial_setup/starter_projects/Nvidia Remix.json (1)
2038-2058: Fail-safe mode input: good addition; align required flags.The BoolInput is clear. To match runtime checks, also toggle project_id.required for IBM in update_build_config when fail_safe_mode flips (on=false → required; on=true → optional), same as you do for api_key.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
src/backend/base/langflow/initial_setup/starter_projects/Nvidia Remix.json(3 hunks)src/lfx/src/lfx/components/elastic/opensearch_multimodal.py(14 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
src/lfx/src/lfx/components/elastic/opensearch_multimodal.py (1)
src/lfx/src/lfx/custom/custom_component/component.py (1)
log(1523-1540)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (65)
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 6/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 48/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 38/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 45/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 41/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 46/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 40/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 37/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 23/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 50/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 44/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 42/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 47/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 34/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 36/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 35/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 27/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 31/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 39/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 49/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 43/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 29/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 33/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 28/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 32/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 25/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 30/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 21/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 26/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 22/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 20/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 24/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 18/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 19/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 17/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 10/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 11/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 15/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 16/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 14/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 12/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 4/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 13/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 9/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 3/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 8/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 7/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 5/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 2/50
- GitHub Check: Run Frontend Tests / Playwright Tests - Shard 1/50
- GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 5
- GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 3
- GitHub Check: Test Docker Images / Test docker images
- GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 4
- GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 2
- GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 1
- GitHub Check: Run Backend Tests / Integration Tests - Python 3.10
- GitHub Check: Run Backend Tests / LFX Tests - Python 3.10
- GitHub Check: Lint Backend / Run Mypy (3.11)
- GitHub Check: Lint Backend / Run Mypy (3.12)
- GitHub Check: Test Starter Templates
- GitHub Check: Optimize new Python code in this PR
- GitHub Check: Run Ruff Check and Format
- GitHub Check: Update Component Index
- GitHub Check: test-starter-projects
🔇 Additional comments (8)
src/lfx/src/lfx/components/elastic/opensearch_multimodal.py (7)
649-650: LGTM! Improved log clarity.Removing the emoji and using consistent debug-level logging improves log parsing and maintains appropriate verbosity levels for mode detection.
684-684: LGTM! Consistent logging format.These changes standardize log messages by removing emojis and using clear, parseable text. This improves consistency across the component and makes logs easier to process programmatically.
Also applies to: 815-815, 983-984
1080-1107: Excellent defensive logging in model detection.The enhanced logging in
_detect_available_modelsprovides valuable diagnostic information:
- Query logging helps debug aggregation issues
- Warning with total document count helps identify whether the index is empty or documents lack the
embedding_modelfield- Fallback behavior is now transparent to operators
This will significantly improve troubleshooting of model detection issues.
1380-1387: LGTM! Enhanced error context.The improved error message provides clear, actionable information when embedding generation fails, including both the models in the index and the available embedding identifiers. This will significantly reduce debugging time.
1416-1416: Excellent validation logging.These logs surface critical validation failures that would otherwise be opaque to users:
- Non-knn_vector field detection prevents runtime errors
- Dimension mismatch detection prevents cryptic OpenSearch errors
- Empty query list provides clear context for why search returned no results
This defensive logging will greatly improve the user experience.
Also applies to: 1427-1429, 1466-1470
1631-1631: Verify search query logging doesn't expose sensitive data.Logging the search query at the start of
search_documentsis helpful for debugging, but ensure that search queries in your use case don't contain sensitive user information (PII, credentials, etc.).If search queries may contain sensitive data, consider:
- Adding a flag to enable/disable query logging
- Truncating long queries
- Redacting patterns that look like sensitive data (emails, SSNs, etc.)
Otherwise, this is a valuable addition for tracing search operations.
1594-1603: LGTM! Valuable empty result debugging.The debug information logged when no results are found (lines 1596-1603) provides excellent context for troubleshooting, including models in index, matched models, KNN fields, and filter counts. This will help users understand why their search returned empty results.
src/backend/base/langflow/initial_setup/starter_projects/Nvidia Remix.json (1)
1856-1856: LGTM: metadata code hash bump.The code_hash update looks correct for this starter project snapshot.
Comment on lines
+2018
to
2019
| "value": "from typing import Any\n\nimport requests\nfrom ibm_watsonx_ai.metanames import EmbedTextParamsMetaNames\nfrom langchain_openai import OpenAIEmbeddings\n\nfrom lfx.base.embeddings.embeddings_class import EmbeddingsWithModels\nfrom lfx.base.embeddings.model import LCEmbeddingsModel\nfrom lfx.base.models.model_utils import get_ollama_models, is_valid_ollama_url\nfrom lfx.base.models.openai_constants import OPENAI_EMBEDDING_MODEL_NAMES\nfrom lfx.base.models.watsonx_constants import (\n IBM_WATSONX_URLS,\n WATSONX_EMBEDDING_MODEL_NAMES,\n)\nfrom lfx.field_typing import Embeddings\nfrom lfx.io import (\n BoolInput,\n DictInput,\n DropdownInput,\n FloatInput,\n IntInput,\n MessageTextInput,\n SecretStrInput,\n)\nfrom lfx.log.logger import logger\nfrom lfx.schema.dotdict import dotdict\nfrom lfx.utils.util import transform_localhost_url\n\n# Ollama API constants\nHTTP_STATUS_OK = 200\nJSON_MODELS_KEY = \"models\"\nJSON_NAME_KEY = \"name\"\nJSON_CAPABILITIES_KEY = \"capabilities\"\nDESIRED_CAPABILITY = \"embedding\"\nDEFAULT_OLLAMA_URL = \"http://localhost:11434\"\n\n\nclass EmbeddingModelComponent(LCEmbeddingsModel):\n display_name = \"Embedding Model\"\n description = \"Generate embeddings using a specified provider.\"\n documentation: str = \"https://docs.langflow.org/components-embedding-models\"\n icon = \"binary\"\n name = \"EmbeddingModel\"\n category = \"models\"\n\n inputs = [\n DropdownInput(\n name=\"provider\",\n display_name=\"Model Provider\",\n options=[\"OpenAI\", \"Ollama\", \"IBM watsonx.ai\"],\n value=\"OpenAI\",\n info=\"Select the embedding model provider\",\n real_time_refresh=True,\n options_metadata=[{\"icon\": \"OpenAI\"}, {\"icon\": \"Ollama\"}, {\"icon\": \"WatsonxAI\"}],\n ),\n MessageTextInput(\n name=\"api_base\",\n display_name=\"API Base URL\",\n info=\"Base URL for the API. Leave empty for default.\",\n advanced=True,\n ),\n MessageTextInput(\n name=\"ollama_base_url\",\n display_name=\"Ollama API URL\",\n info=f\"Endpoint of the Ollama API (Ollama only). Defaults to {DEFAULT_OLLAMA_URL}\",\n value=DEFAULT_OLLAMA_URL,\n show=False,\n real_time_refresh=True,\n load_from_db=True,\n ),\n DropdownInput(\n name=\"base_url_ibm_watsonx\",\n display_name=\"watsonx API Endpoint\",\n info=\"The base URL of the API (IBM watsonx.ai only)\",\n options=IBM_WATSONX_URLS,\n value=IBM_WATSONX_URLS[0],\n show=False,\n real_time_refresh=True,\n ),\n DropdownInput(\n name=\"model\",\n display_name=\"Model Name\",\n options=OPENAI_EMBEDDING_MODEL_NAMES,\n value=OPENAI_EMBEDDING_MODEL_NAMES[0],\n info=\"Select the embedding model to use\",\n real_time_refresh=True,\n refresh_button=True,\n ),\n SecretStrInput(\n name=\"api_key\",\n display_name=\"OpenAI API Key\",\n info=\"Model Provider API key\",\n required=True,\n show=True,\n real_time_refresh=True,\n ),\n # Watson-specific inputs\n MessageTextInput(\n name=\"project_id\",\n display_name=\"Project ID\",\n info=\"IBM watsonx.ai Project ID (required for IBM watsonx.ai)\",\n show=False,\n ),\n IntInput(\n name=\"dimensions\",\n display_name=\"Dimensions\",\n info=\"The number of dimensions the resulting output embeddings should have. \"\n \"Only supported by certain models.\",\n advanced=True,\n ),\n IntInput(name=\"chunk_size\", display_name=\"Chunk Size\", advanced=True, value=1000),\n FloatInput(name=\"request_timeout\", display_name=\"Request Timeout\", advanced=True),\n IntInput(name=\"max_retries\", display_name=\"Max Retries\", advanced=True, value=3),\n BoolInput(name=\"show_progress_bar\", display_name=\"Show Progress Bar\", advanced=True),\n DictInput(\n name=\"model_kwargs\",\n display_name=\"Model Kwargs\",\n advanced=True,\n info=\"Additional keyword arguments to pass to the model.\",\n ),\n IntInput(\n name=\"truncate_input_tokens\",\n display_name=\"Truncate Input Tokens\",\n advanced=True,\n value=200,\n show=False,\n ),\n BoolInput(\n name=\"input_text\",\n display_name=\"Include the original text in the output\",\n value=True,\n advanced=True,\n show=False,\n ),\n BoolInput(\n name=\"fail_safe_mode\",\n display_name=\"Fail-Safe Mode\",\n value=False,\n advanced=True,\n info=\"When enabled, errors will be logged instead of raising exceptions. \"\n \"The component will return None on error.\",\n real_time_refresh=True,\n ),\n ]\n\n @staticmethod\n def fetch_ibm_models(base_url: str) -> list[str]:\n \"\"\"Fetch available models from the watsonx.ai API.\"\"\"\n try:\n endpoint = f\"{base_url}/ml/v1/foundation_model_specs\"\n params = {\n \"version\": \"2024-09-16\",\n \"filters\": \"function_embedding,!lifecycle_withdrawn:and\",\n }\n response = requests.get(endpoint, params=params, timeout=10)\n response.raise_for_status()\n data = response.json()\n models = [model[\"model_id\"] for model in data.get(\"resources\", [])]\n return sorted(models)\n except Exception: # noqa: BLE001\n logger.exception(\"Error fetching models\")\n return WATSONX_EMBEDDING_MODEL_NAMES\n\n async def fetch_ollama_models(self) -> list[str]:\n try:\n return await get_ollama_models(\n base_url_value=self.ollama_base_url,\n desired_capability=DESIRED_CAPABILITY,\n json_models_key=JSON_MODELS_KEY,\n json_name_key=JSON_NAME_KEY,\n json_capabilities_key=JSON_CAPABILITIES_KEY,\n )\n except Exception: # noqa: BLE001\n logger.exception(\"Error fetching models\")\n return []\n\n async def build_embeddings(self) -> Embeddings:\n provider = self.provider\n model = self.model\n api_key = self.api_key\n api_base = self.api_base\n base_url_ibm_watsonx = self.base_url_ibm_watsonx\n ollama_base_url = self.ollama_base_url\n dimensions = self.dimensions\n chunk_size = self.chunk_size\n request_timeout = self.request_timeout\n max_retries = self.max_retries\n show_progress_bar = self.show_progress_bar\n model_kwargs = self.model_kwargs or {}\n\n if provider == \"OpenAI\":\n if not api_key:\n msg = \"OpenAI API key is required when using OpenAI provider\"\n if self.fail_safe_mode:\n logger.error(msg)\n return None\n raise ValueError(msg)\n\n try:\n # Create the primary embedding instance\n embeddings_instance = OpenAIEmbeddings(\n model=model,\n dimensions=dimensions or None,\n base_url=api_base or None,\n api_key=api_key,\n chunk_size=chunk_size,\n max_retries=max_retries,\n timeout=request_timeout or None,\n show_progress_bar=show_progress_bar,\n model_kwargs=model_kwargs,\n )\n\n # Create dedicated instances for each available model\n available_models_dict = {}\n for model_name in OPENAI_EMBEDDING_MODEL_NAMES:\n available_models_dict[model_name] = OpenAIEmbeddings(\n model=model_name,\n dimensions=dimensions or None, # Use same dimensions config for all\n base_url=api_base or None,\n api_key=api_key,\n chunk_size=chunk_size,\n max_retries=max_retries,\n timeout=request_timeout or None,\n show_progress_bar=show_progress_bar,\n model_kwargs=model_kwargs,\n )\n\n return EmbeddingsWithModels(\n embeddings=embeddings_instance,\n available_models=available_models_dict,\n )\n except Exception as e:\n msg = f\"Failed to initialize OpenAI embeddings: {e}\"\n if self.fail_safe_mode:\n logger.error(msg)\n return None\n raise\n\n if provider == \"Ollama\":\n try:\n from langchain_ollama import OllamaEmbeddings\n except ImportError:\n try:\n from langchain_community.embeddings import OllamaEmbeddings\n except ImportError:\n msg = \"Please install langchain-ollama: pip install langchain-ollama\"\n if self.fail_safe_mode:\n logger.error(msg)\n return None\n raise ImportError(msg) from None\n\n try:\n transformed_base_url = transform_localhost_url(ollama_base_url)\n\n # Check if URL contains /v1 suffix (OpenAI-compatible mode)\n if transformed_base_url and transformed_base_url.rstrip(\"/\").endswith(\"/v1\"):\n # Strip /v1 suffix and log warning\n transformed_base_url = transformed_base_url.rstrip(\"/\").removesuffix(\"/v1\")\n logger.warning(\n \"Detected '/v1' suffix in base URL. The Ollama component uses the native Ollama API, \"\n \"not the OpenAI-compatible API. The '/v1' suffix has been automatically removed. \"\n \"If you want to use the OpenAI-compatible API, please use the OpenAI component instead. \"\n \"Learn more at https://docs.ollama.com/openai#openai-compatibility\"\n )\n\n final_base_url = transformed_base_url or \"http://localhost:11434\"\n\n # Create the primary embedding instance\n embeddings_instance = OllamaEmbeddings(\n model=model,\n base_url=final_base_url,\n **model_kwargs,\n )\n\n # Fetch available Ollama models\n available_model_names = await self.fetch_ollama_models()\n\n # Create dedicated instances for each available model\n available_models_dict = {}\n for model_name in available_model_names:\n available_models_dict[model_name] = OllamaEmbeddings(\n model=model_name,\n base_url=final_base_url,\n **model_kwargs,\n )\n\n return EmbeddingsWithModels(\n embeddings=embeddings_instance,\n available_models=available_models_dict,\n )\n except Exception as e:\n msg = f\"Failed to initialize Ollama embeddings: {e}\"\n if self.fail_safe_mode:\n logger.error(msg)\n return None\n raise\n\n if provider == \"IBM watsonx.ai\":\n try:\n from langchain_ibm import WatsonxEmbeddings\n except ImportError:\n msg = \"Please install langchain-ibm: pip install langchain-ibm\"\n if self.fail_safe_mode:\n logger.error(msg)\n return None\n raise ImportError(msg) from None\n\n if not api_key:\n msg = \"IBM watsonx.ai API key is required when using IBM watsonx.ai provider\"\n if self.fail_safe_mode:\n logger.error(msg)\n return None\n raise ValueError(msg)\n\n project_id = self.project_id\n\n if not project_id:\n msg = \"Project ID is required for IBM watsonx.ai provider\"\n if self.fail_safe_mode:\n logger.error(msg)\n return None\n raise ValueError(msg)\n\n try:\n from ibm_watsonx_ai import APIClient, Credentials\n\n final_url = base_url_ibm_watsonx or \"https://us-south.ml.cloud.ibm.com\"\n\n credentials = Credentials(\n api_key=self.api_key,\n url=final_url,\n )\n\n api_client = APIClient(credentials)\n\n params = {\n EmbedTextParamsMetaNames.TRUNCATE_INPUT_TOKENS: self.truncate_input_tokens,\n EmbedTextParamsMetaNames.RETURN_OPTIONS: {\"input_text\": self.input_text},\n }\n\n # Create the primary embedding instance\n embeddings_instance = WatsonxEmbeddings(\n model_id=model,\n params=params,\n watsonx_client=api_client,\n project_id=project_id,\n )\n\n # Fetch available IBM watsonx.ai models\n available_model_names = self.fetch_ibm_models(final_url)\n\n # Create dedicated instances for each available model\n available_models_dict = {}\n for model_name in available_model_names:\n available_models_dict[model_name] = WatsonxEmbeddings(\n model_id=model_name,\n params=params,\n watsonx_client=api_client,\n project_id=project_id,\n )\n\n return EmbeddingsWithModels(\n embeddings=embeddings_instance,\n available_models=available_models_dict,\n )\n except Exception as e:\n msg = f\"Failed to authenticate with IBM watsonx.ai: {e}\"\n if self.fail_safe_mode:\n logger.error(msg)\n return None\n raise\n\n msg = f\"Unknown provider: {provider}\"\n if self.fail_safe_mode:\n logger.error(msg)\n return None\n raise ValueError(msg)\n\n async def update_build_config(\n self, build_config: dotdict, field_value: Any, field_name: str | None = None\n ) -> dotdict:\n # Handle fail_safe_mode changes first - set all required fields to False if enabled\n if field_name == \"fail_safe_mode\":\n if field_value: # If fail_safe_mode is enabled\n build_config[\"api_key\"][\"required\"] = False\n elif hasattr(self, \"provider\"):\n # If fail_safe_mode is disabled, restore required flags based on provider\n if self.provider in [\"OpenAI\", \"IBM watsonx.ai\"]:\n build_config[\"api_key\"][\"required\"] = True\n else: # Ollama\n build_config[\"api_key\"][\"required\"] = False\n\n if field_name == \"provider\":\n if field_value == \"OpenAI\":\n build_config[\"model\"][\"options\"] = OPENAI_EMBEDDING_MODEL_NAMES\n build_config[\"model\"][\"value\"] = OPENAI_EMBEDDING_MODEL_NAMES[0]\n build_config[\"api_key\"][\"display_name\"] = \"OpenAI API Key\"\n # Only set required=True if fail_safe_mode is not enabled\n build_config[\"api_key\"][\"required\"] = not (hasattr(self, \"fail_safe_mode\") and self.fail_safe_mode)\n build_config[\"api_key\"][\"show\"] = True\n build_config[\"api_base\"][\"display_name\"] = \"OpenAI API Base URL\"\n build_config[\"api_base\"][\"advanced\"] = True\n build_config[\"api_base\"][\"show\"] = True\n build_config[\"ollama_base_url\"][\"show\"] = False\n build_config[\"project_id\"][\"show\"] = False\n build_config[\"base_url_ibm_watsonx\"][\"show\"] = False\n build_config[\"truncate_input_tokens\"][\"show\"] = False\n build_config[\"input_text\"][\"show\"] = False\n elif field_value == \"Ollama\":\n build_config[\"ollama_base_url\"][\"show\"] = True\n\n if await is_valid_ollama_url(url=self.ollama_base_url):\n try:\n models = await self.fetch_ollama_models()\n build_config[\"model\"][\"options\"] = models\n build_config[\"model\"][\"value\"] = models[0] if models else \"\"\n except ValueError:\n build_config[\"model\"][\"options\"] = []\n build_config[\"model\"][\"value\"] = \"\"\n else:\n build_config[\"model\"][\"options\"] = []\n build_config[\"model\"][\"value\"] = \"\"\n build_config[\"truncate_input_tokens\"][\"show\"] = False\n build_config[\"input_text\"][\"show\"] = False\n build_config[\"api_key\"][\"display_name\"] = \"API Key (Optional)\"\n build_config[\"api_key\"][\"required\"] = False\n build_config[\"api_key\"][\"show\"] = False\n build_config[\"api_base\"][\"show\"] = False\n build_config[\"project_id\"][\"show\"] = False\n build_config[\"base_url_ibm_watsonx\"][\"show\"] = False\n\n elif field_value == \"IBM watsonx.ai\":\n build_config[\"model\"][\"options\"] = self.fetch_ibm_models(base_url=self.base_url_ibm_watsonx)\n build_config[\"model\"][\"value\"] = self.fetch_ibm_models(base_url=self.base_url_ibm_watsonx)[0]\n build_config[\"api_key\"][\"display_name\"] = \"IBM watsonx.ai API Key\"\n # Only set required=True if fail_safe_mode is not enabled\n build_config[\"api_key\"][\"required\"] = not (hasattr(self, \"fail_safe_mode\") and self.fail_safe_mode)\n build_config[\"api_key\"][\"show\"] = True\n build_config[\"api_base\"][\"show\"] = False\n build_config[\"ollama_base_url\"][\"show\"] = False\n build_config[\"base_url_ibm_watsonx\"][\"show\"] = True\n build_config[\"project_id\"][\"show\"] = True\n build_config[\"truncate_input_tokens\"][\"show\"] = True\n build_config[\"input_text\"][\"show\"] = True\n elif field_name == \"base_url_ibm_watsonx\":\n build_config[\"model\"][\"options\"] = self.fetch_ibm_models(base_url=field_value)\n build_config[\"model\"][\"value\"] = self.fetch_ibm_models(base_url=field_value)[0]\n elif field_name == \"ollama_base_url\":\n # # Refresh Ollama models when base URL changes\n # if hasattr(self, \"provider\") and self.provider == \"Ollama\":\n # Use field_value if provided, otherwise fall back to instance attribute\n ollama_url = self.ollama_base_url\n if await is_valid_ollama_url(url=ollama_url):\n try:\n models = await self.fetch_ollama_models()\n build_config[\"model\"][\"options\"] = models\n build_config[\"model\"][\"value\"] = models[0] if models else \"\"\n except ValueError:\n await logger.awarning(\"Failed to fetch Ollama embedding models.\")\n build_config[\"model\"][\"options\"] = []\n build_config[\"model\"][\"value\"] = \"\"\n\n elif field_name == \"model\" and self.provider == \"Ollama\":\n ollama_url = self.ollama_base_url\n if await is_valid_ollama_url(url=ollama_url):\n try:\n models = await self.fetch_ollama_models()\n build_config[\"model\"][\"options\"] = models\n except ValueError:\n await logger.awarning(\"Failed to refresh Ollama embedding models.\")\n build_config[\"model\"][\"options\"] = []\n\n return build_config\n" | ||
| }, |
Contributor
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# First, let's verify the file exists and check its structure
head -20 "src/backend/base/langflow/initial_setup/starter_projects/Nvidia Remix.json"Repository: langflow-ai/langflow
Length of output: 537
🏁 Script executed:
# Check file size to understand what we're working with
wc -l "src/backend/base/langflow/initial_setup/starter_projects/Nvidia Remix.json"Repository: langflow-ai/langflow
Length of output: 143
🏁 Script executed:
# Extract lines around 2018-2019 to see the context
sed -n '2000,2100p' "src/backend/base/langflow/initial_setup/starter_projects/Nvidia Remix.json" | head -100Repository: langflow-ai/langflow
Length of output: 24862
🏁 Script executed:
# Extract the Python code value from the JSON by parsing it
python3 << 'EOF'
import json
import re
# Read the JSON file
with open("src/backend/base/langflow/initial_setup/starter_projects/Nvidia Remix.json", "r") as f:
data = json.load(f)
# Navigate to the code value - it should be in nodes
for node in data.get("data", {}).get("nodes", []):
if "data" in node and "code" in node["data"]:
code_value = node["data"]["code"].get("value", "")
if "update_build_config" in code_value:
# Find the ollama_base_url handler section
lines = code_value.split('\n')
for i, line in enumerate(lines):
if "elif field_name == \"ollama_base_url\"" in line:
# Print 20 lines starting from this handler
print(f"\n=== OLLAMA_BASE_URL HANDLER (lines {i}-{i+20}) ===")
for j in range(i, min(i+25, len(lines))):
print(f"{j}: {lines[j]}")
if "elif field_value == \"IBM watsonx.ai\"" in line:
# Print IBM handler
print(f"\n=== IBM WATSONX HANDLER (lines {i}-{i+15}) ===")
for j in range(i, min(i+20, len(lines))):
print(f"{j}: {lines[j]}")
# Check return type annotation of build_embeddings
for i, line in enumerate(lines):
if "async def build_embeddings" in line:
print(f"\n=== BUILD_EMBEDDINGS RETURN TYPE (lines {i}-{i+2}) ===")
for j in range(i, min(i+2, len(lines))):
print(f"{j}: {lines[j]}")
break
EOFRepository: langflow-ai/langflow
Length of output: 150
🏁 Script executed:
# Use sed to extract the relevant sections from around line 2018
# Focus on the update_build_config method
sed -n '4700,4850p' "src/backend/base/langflow/initial_setup/starter_projects/Nvidia Remix.json"Repository: langflow-ai/langflow
Length of output: 8029
🏁 Script executed:
# Find lines containing ollama_base_url handler
rg -n "elif field_name == \"ollama_base_url\"" "src/backend/base/langflow/initial_setup/starter_projects/Nvidia Remix.json"Repository: langflow-ai/langflow
Length of output: 46
🏁 Script executed:
# Find where ollama_base_url appears in the file
rg -n "ollama_base_url" "src/backend/base/langflow/initial_setup/starter_projects/Nvidia Remix.json" | head -20Repository: langflow-ai/langflow
Length of output: 21301
🏁 Script executed:
# Extract just the relevant sections to verify each concern
cat > /tmp/verify.py << 'EOF'
import json
with open("src/backend/base/langflow/initial_setup/starter_projects/Nvidia Remix.json", "r") as f:
data = json.load(f)
# Find the code node
for node in data.get("data", {}).get("nodes", []):
if "data" in node and "code" in node["data"]:
code = node["data"]["code"].get("value", "")
if "fetch_ollama_models" in code:
lines = code.split('\n')
# Check 1: fetch_ollama_models implementation
print("=== CONCERN 1: fetch_ollama_models uses self.ollama_base_url ===")
for i, line in enumerate(lines):
if "async def fetch_ollama_models" in line:
for j in range(i, min(i+10, len(lines))):
print(f"{j}: {lines[j]}")
break
# Check 2: Return type of build_embeddings
print("\n=== CONCERN 2: build_embeddings return type ===")
for i, line in enumerate(lines):
if "async def build_embeddings" in line:
print(f"{i}: {lines[i]}")
break
# Check 3: project_id input definition
print("\n=== CONCERN 3: project_id input definition ===")
for i, line in enumerate(lines):
if 'name="project_id"' in line:
for j in range(max(0, i-2), min(i+8, len(lines))):
print(f"{j}: {lines[j]}")
break
# Check 4: IBM models fetch calls
print("\n=== CONCERN 4: IBM models double fetch ===")
for i, line in enumerate(lines):
if 'elif field_value == "IBM watsonx.ai"' in line:
for j in range(i, min(i+8, len(lines))):
print(f"{j}: {lines[j]}")
break
break
EOF
python3 /tmp/verify.pyRepository: langflow-ai/langflow
Length of output: 46
🏁 Script executed:
# Let me extract the specific problematic code sections directly
# The code is on line 2018 as a JSON value, let me search within it more carefully
# First, let's look for the exact patterns in the problematic sections
sed -n '2018,2019p' "src/backend/base/langflow/initial_setup/starter_projects/Nvidia Remix.json" | \
grep -o "async def fetch_ollama_models.*\}" | head -1 | sed 's/\\n/\n/g' | head -20Repository: langflow-ai/langflow
Length of output: 848
🏁 Script executed:
# Extract IBM watsonx.ai handler to verify double fetch issue
sed -n '2018,2019p' "src/backend/base/langflow/initial_setup/starter_projects/Nvidia Remix.json" | \
grep -o 'elif field_value == "IBM watsonx.ai":.*elif field_name == "base_url_ibm_watsonx"' | \
sed 's/\\n/\n/g' | head -30Repository: langflow-ai/langflow
Length of output: 46
🏁 Script executed:
# Extract the complete ollama_base_url handler section
sed -n '2018,2019p' "src/backend/base/langflow/initial_setup/starter_projects/Nvidia Remix.json" | \
python3 << 'PYSCRIPT'
import sys
content = sys.stdin.read()
# Find ollama_base_url handler
start = content.find('elif field_name == "ollama_base_url":')
if start != -1:
# Extract next 500 chars to see the handler
section = content[start:start+800]
# Unescape the newlines
section = section.replace('\\n', '\n')
print("=== OLLAMA_BASE_URL HANDLER ===")
print(section)
PYSCRIPTRepository: langflow-ai/langflow
Length of output: 46
Ollama base URL handler ignores field_value—model refresh fails when URL is changed in UI.
In update_build_config, the ollama_base_url branch reads self.ollama_base_url instead of field_value, causing model refresh to skip the newly entered URL unless the instance attribute is already updated. Pass field_value to the model fetch:
elif field_name == "ollama_base_url":
- ollama_url = self.ollama_base_url
+ ollama_url = field_value or self.ollama_base_url
if await is_valid_ollama_url(url=ollama_url):Additional issues:
- Double IBM network calls:
self.fetch_ibm_models()is called twice when provider changes to IBM watsonx.ai (once for options, once for value). Cache the result. - Project ID should be conditionally required: Set
build_config["project_id"]["required"] = not (hasattr(self, "fail_safe_mode") and self.fail_safe_mode)to match the runtime validation inbuild_embeddings. - Inaccurate error message: "Failed to authenticate with IBM watsonx.ai" should be "Failed to initialize IBM watsonx.ai embeddings" since errors can occur during client initialization, not just authentication.
- Return type annotation mismatch:
build_embeddingsis annotated as-> Embeddingsbut returnsNonewhenfail_safe_modeis enabled. Change to-> Embeddings | Noneand ensure downstream components check forNone.
🤖 Prompt for AI Agents
In src/backend/base/langflow/initial_setup/starter_projects/Nvidia Remix.json
around lines 2018-2019: the ollama_base_url branch of update_build_config uses
self.ollama_base_url instead of the incoming field_value (so UI changes don't
refresh models); change to use field_value (or prefer field_value when present)
when validating and fetching models. Also, when provider == "IBM watsonx.ai"
avoid calling self.fetch_ibm_models() twice by calling it once, storing the
result and using it for both model options and default value; set
build_config["project_id"]["required"] = not (hasattr(self, "fail_safe_mode")
and self.fail_safe_mode) to match runtime validation; change the IBM exception
message from "Failed to authenticate with IBM watsonx.ai" to "Failed to
initialize IBM watsonx.ai embeddings"; and update the build_embeddings signature
return annotation from -> Embeddings to -> Embeddings | None (and ensure callers
handle None).
github-actions
Bot
added
bug
github-actions
Bot
added
bug
…i/langflow into fix-opensearch-search
github-actions
Bot
added
bug
github-actions
Bot
added
bug
github-actions
Bot
added
bug
github-actions
Bot
added
bug
Contributor
There was a problem hiding this comment.
Pull request overview
This PR enhances error handling and diagnostic logging for the OpenSearch multimodal component's embedding model management. The changes focus on improving model detection resilience and providing better visibility into search operations through comprehensive logging.
Key Changes:
- Enhanced model detection with detailed diagnostics when models are unavailable
- Added extensive debug logging throughout the search pipeline to track model matching and embedding generation
- Improved error messages with contextual information about available models and identifiers
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| model_name = getattr(emb_obj, "model_name", None) | ||
| dimensions = getattr(emb_obj, "dimensions", None) | ||
| available_models = getattr(emb_obj, "available_models", None) | ||
| available_models_attr = getattr(emb_obj, "available_models", None) |
There was a problem hiding this comment.
[nitpick] The variable name 'available_models_attr' is verbose. Consider renaming to 'available_models_dict' to better convey that it's specifically the dictionary attribute, or simply keep 'available_models' if there's no naming conflict in this scope.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request makes a minor update to the
Nvidia Remix.jsonstarter project by changing thecode_hashvalue in the metadata section.code_hashin themetadataofsrc/backend/base/langflow/initial_setup/starter_projects/Nvidia Remix.jsonto reflect the latest code version.Summary by CodeRabbit
New Features
Improvements
✏️ Tip: You can customize this high-level summary in your review settings.