fix: Support Batch Run with watsonX

[erichare]

This pull request improves the robustness of batch LLM operations by addressing issues related to model configuration failures, particularly those caused by serialization problems with sensitive data types like SecretStr. The changes ensure that batch processing can proceed smoothly even if model configuration fails, and adds targeted tests to verify this behavior.

Error Handling Improvements:

• Updated run_batch in batch_run.py to gracefully handle exceptions when calling with_config() on models (such as ChatWatsonx) that may have serialization issues with attributes like SecretStr. If configuration fails, a warning is logged and batch processing continues without configuration.

Sensitive Data Handling:

• Modified build_model in watsonx.py to always pass the API key as a plain string, avoiding the use of SecretStr in a way that could cause serialization errors during model configuration or batch operations.

Testing:

• Added a new test test_with_config_failure_handling to verify that batch runs complete successfully even if a model's with_config() method raises an exception due to serialization errors.

Summary by CodeRabbit

• Bug Fixes

  • Batch processing now gracefully handles model configuration failures without interrupting operations
  • Improved API key serialization handling for consistency across configurations

• Tests

  • Added test coverage verifying batch operations remain resilient when model configuration encounters errors

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Walkthrough

Changes add graceful error handling for batch LLM operations, including a test verifying BatchRunComponent handles model configuration failures, implementing try-catch logic in batch_run.py to continue processing when with_config() fails, and updating API key handling in WatsonxAIComponent to resolve SecretStr values before passing to ChatWatsonx.

Changes

Cohort / File(s)	Summary
Batch operation error handling src/backend/tests/unit/components/llm_operations/test_batch_run_component.py, src/lfx/src/lfx/components/llm_operations/batch_run.py	Adds test test_with_config_failure_handling to verify BatchRunComponent gracefully handles models whose with_config() raises errors. Updates batch_run.py to wrap model configuration in try-except, logging warnings and continuing processing when configuration fails instead of crashing.
API key handling src/lfx/src/lfx/components/ibm/watsonx.py	Resolves API key from SecretStr to plain string before passing to ChatWatsonx constructor. Extracts secret value when key is SecretStr, otherwise uses value as-is, ensuring consistent and serialization-safe API key handling in configurations.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

• Review error handling logic and exception flow in batch_run.py for potential edge cases
• Verify test coverage adequately exercises the with_config() failure scenario
• Confirm API key resolution logic handles both SecretStr and non-SecretStr inputs correctly

Suggested labels

size:L, lgtm

Suggested reviewers

• jordanrfrazier

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%.	You can run @coderabbitai generate docstrings to improve docstring coverage.

✅ Passed checks (6 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'fix: Support Batch Run with watsonX' accurately reflects the main objective of the PR—enabling watsonX model support in batch operations by fixing serialization and configuration issues.
Test Coverage For New Implementations	✅ Passed	PR includes substantive regression test validating core bug fix and end-to-end batch processing behavior with proper DataFrame output despite configuration failures.
Test Quality And Coverage	✅ Passed	Test directly validates error handling when with_config() raises ValueError, uses proper async patterns with await, includes 4 behavioral assertions, follows pytest backend conventions, and tests error path explicitly.
Test File Naming And Structure	✅ Passed	The test file test_batch_run_component.py comprehensively meets all custom check requirements for backend test structure and naming, following test_*.py convention with proper pytest structure, fixtures, class-based tests, 14 well-organized descriptive test methods, and comprehensive coverage including positive scenarios, negative scenarios with error handling, and edge cases.
Excessive Mock Usage Warning	✅ Passed	The pull request demonstrates appropriate and minimal mock usage with real objects and selective test doubles for testing external dependencies, preserving actual implementations and verifying control flow.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

src/lfx/src/lfx/components/llm_operations/batch_run.py (1)

162-177: Graceful error handling for model configuration.

The try-catch block appropriately handles serialization failures during with_config() calls, allowing batch processing to continue. The warning message provides good visibility.

However, Ruff flags the bare Exception catch (BLE001). While this broad catch is justified for handling diverse model implementations, consider adding a noqa: BLE001 comment to acknowledge this is intentional.

Apply this diff to suppress the static analysis warning:

-            except Exception as e:
+            except Exception as e:  # noqa: BLE001
                 # Log warning and continue without configuration

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d8b979d and 8ec2956.

📒 Files selected for processing (3)

• src/backend/tests/unit/components/llm_operations/test_batch_run_component.py (1 hunks)
• src/lfx/src/lfx/components/ibm/watsonx.py (1 hunks)
• src/lfx/src/lfx/components/llm_operations/batch_run.py (1 hunks)

🧰 Additional context used

📓 Path-based instructions (5)

src/backend/**/*.py

📄 CodeRabbit inference engine (.cursor/rules/backend_development.mdc)

src/backend/**/*.py: Use FastAPI async patterns with await for async operations in component execution methods
Use asyncio.create_task() for background tasks and implement proper cleanup with try/except for asyncio.CancelledError
Use queue.put_nowait() for non-blocking queue operations and asyncio.wait_for() with timeouts for controlled get operations

Files:

• src/backend/tests/unit/components/llm_operations/test_batch_run_component.py

src/backend/**/*component*.py

📄 CodeRabbit inference engine (.cursor/rules/icons.mdc)

In Python component classes, set the icon attribute to a string matching the desired icon name (e.g., icon = "AstraDB"). The string must match the frontend icon mapping exactly (case-sensitive).

Files:

• src/backend/tests/unit/components/llm_operations/test_batch_run_component.py

src/backend/tests/**/*.py

📄 CodeRabbit inference engine (.cursor/rules/testing.mdc)

src/backend/tests/**/*.py: Place backend unit tests in src/backend/tests/ directory, component tests in src/backend/tests/unit/components/ organized by component subdirectory, and integration tests accessible via make integration_tests
Use same filename as component with appropriate test prefix/suffix (e.g., my_component.py → test_my_component.py)
Use the client fixture (FastAPI Test Client) defined in src/backend/tests/conftest.py for API tests; it provides an async httpx.AsyncClient with automatic in-memory SQLite database and mocked environment variables. Skip client creation by marking test with @pytest.mark.noclient
Inherit from the correct ComponentTestBase family class located in src/backend/tests/base.py based on API access needs: ComponentTestBase (no API), ComponentTestBaseWithClient (needs API), or ComponentTestBaseWithoutClient (pure logic). Provide three required fixtures: component_class, default_kwargs, and file_names_mapping
Create comprehensive unit tests for all new backend components. If unit tests are incomplete, create a corresponding Markdown file documenting manual testing steps and expected outcomes
Test both sync and async code paths, mock external dependencies appropriately, test error handling and edge cases, validate input/output behavior, and test component initialization and configuration
Use @pytest.mark.asyncio decorator for async component tests and ensure async methods are properly awaited
Test background tasks using asyncio.create_task() and verify completion with asyncio.wait_for() with appropriate timeout constraints
Test queue operations using non-blocking queue.put_nowait() and asyncio.wait_for(queue.get(), timeout=...) to verify queue processing without blocking
Use @pytest.mark.no_blockbuster marker to skip the blockbuster plugin in specific tests
For database tests that may fail in batch runs, run them sequentially using uv run pytest src/backend/tests/unit/test_database.py r...

Files:

• src/backend/tests/unit/components/llm_operations/test_batch_run_component.py

**/{test_*.py,*.test.ts,*.test.tsx}

📄 CodeRabbit inference engine (Custom checks)

Check that test files follow the project's naming conventions (test_*.py for backend, *.test.ts for frontend)

Files:

• src/backend/tests/unit/components/llm_operations/test_batch_run_component.py

**/test_*.py

📄 CodeRabbit inference engine (Custom checks)

**/test_*.py: Backend tests should follow pytest structure with proper test_*.py naming
For async functions, ensure proper async testing patterns are used with pytest for backend

Files:

• src/backend/tests/unit/components/llm_operations/test_batch_run_component.py

🧠 Learnings (4)

📚 Learning: 2025-11-24T19:47:28.997Z

Learnt from: CR
Repo: langflow-ai/langflow PR: 0
File: .cursor/rules/testing.mdc:0-0
Timestamp: 2025-11-24T19:47:28.997Z
Learning: Applies to src/backend/tests/**/*.py : Test component versioning and backward compatibility using `file_names_mapping` fixture with `VersionComponentMapping` objects mapping component files across Langflow versions

Applied to files:

• src/backend/tests/unit/components/llm_operations/test_batch_run_component.py

📚 Learning: 2025-11-24T19:47:28.997Z

Learnt from: CR
Repo: langflow-ai/langflow PR: 0
File: .cursor/rules/testing.mdc:0-0
Timestamp: 2025-11-24T19:47:28.997Z
Learning: Applies to src/backend/tests/**/*.py : Test component build config updates by calling `to_frontend_node()` to get the node template, then calling `update_build_config()` to apply configuration changes

Applied to files:

• src/backend/tests/unit/components/llm_operations/test_batch_run_component.py

📚 Learning: 2025-11-24T19:47:28.997Z

Learnt from: CR
Repo: langflow-ai/langflow PR: 0
File: .cursor/rules/testing.mdc:0-0
Timestamp: 2025-11-24T19:47:28.997Z
Learning: Applies to src/backend/tests/**/*.py : Inherit from the correct `ComponentTestBase` family class located in `src/backend/tests/base.py` based on API access needs: `ComponentTestBase` (no API), `ComponentTestBaseWithClient` (needs API), or `ComponentTestBaseWithoutClient` (pure logic). Provide three required fixtures: `component_class`, `default_kwargs`, and `file_names_mapping`

Applied to files:

• src/backend/tests/unit/components/llm_operations/test_batch_run_component.py

📚 Learning: 2025-11-24T19:47:28.997Z

Learnt from: CR
Repo: langflow-ai/langflow PR: 0
File: .cursor/rules/testing.mdc:0-0
Timestamp: 2025-11-24T19:47:28.997Z
Learning: Applies to src/backend/tests/**/*.py : Test both sync and async code paths, mock external dependencies appropriately, test error handling and edge cases, validate input/output behavior, and test component initialization and configuration

Applied to files:

• src/backend/tests/unit/components/llm_operations/test_batch_run_component.py

🧬 Code graph analysis (1)

src/lfx/src/lfx/components/llm_operations/batch_run.py (2)

src/backend/tests/unit/components/llm_operations/test_batch_run_component.py (3)

• with_config (102-103)
• with_config (131-132)
• with_config (265-267)

src/lfx/src/lfx/custom/custom_component/component.py (1)

• get_project_name (1518-1521)

🪛 GitHub Actions: Ruff Style Check

src/lfx/src/lfx/components/ibm/watsonx.py

[error] 205-205: ruff check failed. W293: Blank line contains whitespace. Command: uv run --only-dev ruff check --output-format=github .

🪛 GitHub Check: Ruff Style Check (3.13)

src/lfx/src/lfx/components/ibm/watsonx.py

[failure] 205-205: Ruff (W293)
src/lfx/src/lfx/components/ibm/watsonx.py:205:1: W293 Blank line contains whitespace

src/lfx/src/lfx/components/llm_operations/batch_run.py

[failure] 172-172: Ruff (BLE001)
src/lfx/src/lfx/components/llm_operations/batch_run.py:172:20: BLE001 Do not catch blind exception: Exception

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (16)

• GitHub Check: Lint Backend / Run Mypy (3.11)
• GitHub Check: Lint Backend / Run Mypy (3.10)
• GitHub Check: Test Docker Images / Test docker images
• GitHub Check: Lint Backend / Run Mypy (3.13)
• GitHub Check: Lint Backend / Run Mypy (3.12)
• GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 5
• GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 4
• GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 3
• GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 2
• GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 1
• GitHub Check: Run Backend Tests / LFX Tests - Python 3.10
• GitHub Check: Run Backend Tests / Integration Tests - Python 3.10
• GitHub Check: Test Starter Templates
• GitHub Check: Update Component Index
• GitHub Check: Run Ruff Check and Format
• GitHub Check: Update Starter Projects

🔇 Additional comments (2)

src/backend/tests/unit/components/llm_operations/test_batch_run_component.py (1)

260-289: Excellent test coverage for config failure handling.

This test effectively validates the new error handling in batch_run.py. The ConfigFailureModel accurately simulates the SecretStr serialization failure scenario, and the assertions confirm that batch processing completes successfully despite the with_config() failure.

The test structure follows best practices with a clear docstring, proper async patterns, and comprehensive assertions.

src/lfx/src/lfx/components/ibm/watsonx.py (1)

200-204: Correct API key handling to prevent serialization issues.

The change to extract the plain string value from SecretStr before passing to ChatWatsonx is essential for avoiding serialization failures during with_config() and batch operations. The defensive check handles both SecretStr and plain string inputs appropriately.

The explanatory comment is helpful for future maintainers.

Also applies to: 207-207

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 32.54%. Comparing base (fbfa529) to head (b542ac1).
⚠️ Report is 5 commits behind head on main.

❌ Your project status has failed because the head coverage (39.99%) is below the target coverage (60.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files

@@           Coverage Diff           @@
##             main   #10848   +/-   ##
=======================================
  Coverage   32.53%   32.54%           
=======================================
  Files        1370     1370           
  Lines       63518    63518           
  Branches     9391     9391           
=======================================
+ Hits        20668    20670    +2     
+ Misses      41810    41809    -1     
+ Partials     1040     1039    -1     

Flag	Coverage Δ
backend	51.52% <ø> (+<0.01%)	⬆️
frontend	14.28% <ø> (ø)
lfx	39.99% <ø> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 3 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

Frontend Unit Test Coverage Report

Coverage Summary

Lines	Statements	Branches	Functions
	15.43% (4244/27499)	8.61% (1811/21013)	9.69% (587/6057)

Unit Test Results

Tests	Skipped	Failures	Errors	Time
1671	0 💤	0 ❌	0 🔥	21.051s ⏱️