Skip to content

feat: add VLM Run audio and video transcription component#9460

Merged
erichare merged 17 commits into
langflow-ai:mainfrom
Teraflop-Inc:vlm-run-video-audio-integration
Oct 9, 2025
Merged

feat: add VLM Run audio and video transcription component#9460
erichare merged 17 commits into
langflow-ai:mainfrom
Teraflop-Inc:vlm-run-video-audio-integration

Conversation

@wbrennan899
Copy link
Copy Markdown
Contributor

@wbrennan899 wbrennan899 commented Aug 20, 2025

Summary

  • Add new VLM Run AI component for audio and video transcription
  • Support multiple audio/video formats with configurable output options
  • Include comprehensive unit tests and custom icon

What's New

  • VLMRunTranscription component for transcribing audio and video files using VLM Run AI
  • Support for audio formats: MP3, WAV, M4A, FLAC, OGG, OPUS, WEBM, AAC
  • Support for video formats: MP4, MOV, AVI, MKV, WMV, FLV, WEBM, MPG, MPEG, 3GP
  • Configurable transcription models (fast, pro, ultra)
  • Multiple output formats (text, json, srt, vtt)
  • Optional prompt customization for guided transcription
  • Batch processing for multiple files

Changes

  • Add vlmrun package dependency
  • Create VLMRunTranscription component with full configuration options
  • Add comprehensive unit tests with mocked API responses
  • Include custom VLM Run icon with lazy loading support

Testing

All tests pass with 100% coverage for the new component.

Summary by CodeRabbit

  • New Features

    • Introduced VLM Run Transcription component (beta) to transcribe audio/video from files or URLs, with multi-file support and combined results.
    • Added VLM Run icon and a new “VLM Run” sidebar entry for easier discovery.
  • Chores

    • Updated dependencies to include VLM Run SDK.
    • Added an optional Couchbase dependency group for those who need it.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Aug 20, 2025

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Walkthrough

Adds a new VLM Run transcription component to the backend, declares new dependencies, introduces unit tests, and integrates a corresponding icon and sidebar bundle entry in the frontend.

Changes

Cohort / File(s) Summary
Dependency declarations
pyproject.toml
Adds dependency vlmrun[all]>=0.2.0; introduces optional group couchbase with couchbase>=4.2.1.
Backend VLMRun component
src/backend/base/langflow/components/vlmrun/__init__.py, src/backend/base/langflow/components/vlmrun/vlmrun_transcription.py
Exposes new component VLMRunTranscription and implements media transcription via VLM Run SDK for files or URL, with batching, result aggregation, and error handling.
Backend tests
src/backend/tests/unit/components/vlmrun/test_vlmrun_transcription.py
Adds unit tests covering metadata, inputs/outputs, error paths (no input, missing SDK), frontend node data, file type options, and defaults.
Frontend VLMRun UI integration
src/frontend/src/icons/VLMRun/index.tsx, src/frontend/src/icons/eagerIconImports.ts, src/frontend/src/icons/lazyIconImports.ts, src/frontend/src/utils/styleUtils.ts
Adds VLM Run SVG icon, registers it in eager/lazy icon maps, and adds a "VLM Run" sidebar bundle entry with name "vlmrun" and icon "VLMRun".

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant U as User/Flow
  participant C as VLMRunTranscription (Component)
  participant SDK as VLMRun SDK
  participant API as VLM Run Service

  U->>C: process_media(api_key, media_type, files|url)
  alt SDK not installed
    C-->>U: Data{ error: "VLM Run SDK not installed" }
  else Inputs provided
    C->>SDK: new Client(api_key)
    alt Media files
      loop For each file
        C->>SDK: audio/video.generate(..., batch=true)
        SDK->>API: Create transcription job
        API-->>SDK: job id
        C->>SDK: wait_for_batch(job id, timeout)
        SDK-->>C: batch result with segments
        C-->>C: extract and assemble transcription
      end
      C-->>U: Data{ results[], combined_transcription }
    else Media URL
      C->>SDK: audio/video.generate(url, batch=true)
      SDK->>API: Create transcription job
      API-->>SDK: job id
      C->>SDK: wait_for_batch(job id, timeout)
      SDK-->>C: result with segments
      C-->>U: Data{ result }
    end
  end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

Suggested labels

size:M, lgtm

Suggested reviewers

  • ogabrielluiz
  • mfortman11
  • jordanrfrazier
✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Aug 20, 2025
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (8)
pyproject.toml (2)

129-130: Add vlmrun as an optional extra instead of a hard dependency (consider).

Including vlmrun[all]>=0.2.0 in core dependencies may significantly increase install size and transitive deps for users not using VLM Run. Consider moving it to [project.optional-dependencies] (e.g., audio or a new vlmrun extra) and adjusting docs to pip install langflow[vlmrun]. If keeping it mandatory is deliberate, ignore.

-[project]
-dependencies = [
-    ...
-    "vlmrun[all]>=0.2.0",
-]
+[project]
+dependencies = [
+    ...
+    # (optional) consider moving to extras; keep here only if mandatory for baseline
+    # "vlmrun[all]>=0.2.0",
+]
@@
 [project.optional-dependencies]
@@
+vlmrun = [
+    "vlmrun[all]>=0.2.0",
+]

205-208: New optional couchbase extra: add install note and CI gate if applicable.

couchbase>=4.2.1 often requires system libs; if you plan to test this extra in CI, ensure runners have prerequisites. Otherwise, add a brief note in docs (README or installation page) to use pip install langflow[couchbase].

src/backend/tests/unit/components/vlmrun/test_vlmrun_transcription.py (2)

136-144: Align expected video extensions with PR goals

PR description claims support for MPG, MPEG, and 3GP, but the test doesn’t assert them. Either add those extensions to the component or adjust the PR description. If adding support, update the expectations here accordingly.

Proposed test update if the component adds those types:

-        expected_video_types = ["mp4", "mov", "avi", "mkv", "flv", "wmv", "m4v"]
+        expected_video_types = ["mp4", "mov", "avi", "mkv", "flv", "wmv", "m4v", "mpg", "mpeg", "3gp"]

107-131: Consider adding happy-path and multi-file/URL tests with a stubbed SDK

Right now we only test metadata, structure, and error paths. Add tests that:

  • Mock vlmrun.client.VLMRun to return a response object with id, response.segments, usage, and status.
  • Cover: single audio file, single video file (including audio-in-video segments), multiple files (combined_transcription), and URL input.
  • Assert output shape (filename/source, prediction_id, transcription, metadata.duration passthrough, usage/status presence).

I can draft a minimal fake VLMRun client and fixtures to cover these paths. Want me to add them?

src/backend/base/langflow/components/vlmrun/vlmrun_transcription.py (4)

40-56: Add missing declared video formats (mpg, mpeg, 3gp) to match PR feature set

The PR description states support for MPG, MPEG, and 3GP, which are not currently listed. Add them to file_types so the UI accepts those uploads.

Apply this diff:

             file_types=[
                 "mp3",
                 "wav",
                 "m4a",
                 "flac",
                 "ogg",
                 "opus",
                 "webm",
                 "aac",
                 "mp4",
                 "mov",
                 "avi",
                 "mkv",
                 "flv",
                 "wmv",
                 "m4v",
+                "mpg",
+                "mpeg",
+                "3gp",
             ],

78-85: Optional: make this non-blocking in async flows

process_media performs network I/O and can block up to 600s. If this component is used in an async context, consider converting to async def and wrapping blocking SDK calls with asyncio.to_thread to avoid blocking the event loop. Keep a sync wrapper if backward compatibility requires it.

If you choose to go async, verify call sites (flow runner) accept async outputs for component methods.


200-218: Set Data.text for downstream compatibility

Many downstream components expect Data.text to be populated. Mirror transcription into a text key for both single and multi-file results.

Apply this diff:

             if len(all_results) == 1:
                 # Single file/URL - return simple format for backward compatibility
-                output_data = all_results[0]
+                output_data = all_results[0]
             else:
                 # Multiple files - return list of results
                 output_data = {
                     "results": all_results,
                     "total_files": len(all_results),
                     "combined_transcription": "\n\n---\n\n".join(
                         [
                             f"[{r.get('filename', r.get('source', 'Unknown'))}]\n{r['transcription']}"
                             for r in all_results
                         ]
                     ),
                 }
+            # Add a 'text' convenience key for downstream components
+            if isinstance(output_data, dict) and "transcription" in output_data:
+                output_data.setdefault("text", output_data["transcription"])
+            elif isinstance(output_data, dict) and "combined_transcription" in output_data:
+                output_data.setdefault("text", output_data["combined_transcription"])

86-94: Nit: redundant outer ImportError handling

You already catch ImportError in the inner import block; catching ImportError again in the outer try/except is redundant unless other imports below can raise it. Safe to keep; mentioning for clarity only.

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between ea918df and 1e45bbc.

⛔ Files ignored due to path filters (2)
  • src/frontend/src/icons/VLMRun/vlm-blue.svg is excluded by !**/*.svg
  • uv.lock is excluded by !**/*.lock
📒 Files selected for processing (8)
  • pyproject.toml (2 hunks)
  • src/backend/base/langflow/components/vlmrun/__init__.py (1 hunks)
  • src/backend/base/langflow/components/vlmrun/vlmrun_transcription.py (1 hunks)
  • src/backend/tests/unit/components/vlmrun/test_vlmrun_transcription.py (1 hunks)
  • src/frontend/src/icons/VLMRun/index.tsx (1 hunks)
  • src/frontend/src/icons/eagerIconImports.ts (2 hunks)
  • src/frontend/src/icons/lazyIconImports.ts (1 hunks)
  • src/frontend/src/utils/styleUtils.ts (1 hunks)
🧰 Additional context used
📓 Path-based instructions (13)
src/frontend/src/**/*.{ts,tsx,js,jsx}

📄 CodeRabbit Inference Engine (.cursor/rules/frontend_development.mdc)

src/frontend/src/**/*.{ts,tsx,js,jsx}: All frontend TypeScript and JavaScript code should be located under src/frontend/src/ and organized into components, pages, icons, stores, types, utils, hooks, services, and assets directories as per the specified directory layout.
Use React 18 with TypeScript for all UI components in the frontend.
Format all TypeScript and JavaScript code using the make format_frontend command.
Lint all TypeScript and JavaScript code using the make lint command.

Files:

  • src/frontend/src/utils/styleUtils.ts
  • src/frontend/src/icons/lazyIconImports.ts
  • src/frontend/src/icons/eagerIconImports.ts
  • src/frontend/src/icons/VLMRun/index.tsx
src/frontend/src/utils/**/*.{ts,tsx,js,jsx}

📄 CodeRabbit Inference Engine (.cursor/rules/frontend_development.mdc)

All utility functions should be placed in the utils directory.

Files:

  • src/frontend/src/utils/styleUtils.ts
src/backend/base/langflow/components/**/*.py

📄 CodeRabbit Inference Engine (.cursor/rules/backend_development.mdc)

src/backend/base/langflow/components/**/*.py: Add new backend components to the appropriate subdirectory under src/backend/base/langflow/components/
Implement async component methods using async def and await for asynchronous operations
Use asyncio.create_task for background work in async components and ensure proper cleanup on cancellation
Use asyncio.Queue for non-blocking queue operations in async components and handle timeouts appropriately

Files:

  • src/backend/base/langflow/components/vlmrun/__init__.py
  • src/backend/base/langflow/components/vlmrun/vlmrun_transcription.py
src/backend/base/langflow/components/**/__init__.py

📄 CodeRabbit Inference Engine (.cursor/rules/backend_development.mdc)

Update init.py with alphabetical imports when adding new components

Files:

  • src/backend/base/langflow/components/vlmrun/__init__.py
{src/backend/**/*.py,tests/**/*.py,Makefile}

📄 CodeRabbit Inference Engine (.cursor/rules/backend_development.mdc)

{src/backend/**/*.py,tests/**/*.py,Makefile}: Run make format_backend to format Python code before linting or committing changes
Run make lint to perform linting checks on backend Python code

Files:

  • src/backend/base/langflow/components/vlmrun/__init__.py
  • src/backend/base/langflow/components/vlmrun/vlmrun_transcription.py
  • src/backend/tests/unit/components/vlmrun/test_vlmrun_transcription.py
src/backend/**/components/**/*.py

📄 CodeRabbit Inference Engine (.cursor/rules/icons.mdc)

In your Python component class, set the icon attribute to a string matching the frontend icon mapping exactly (case-sensitive).

Files:

  • src/backend/base/langflow/components/vlmrun/__init__.py
  • src/backend/base/langflow/components/vlmrun/vlmrun_transcription.py
  • src/backend/tests/unit/components/vlmrun/test_vlmrun_transcription.py
src/frontend/src/icons/**/*.{ts,tsx,js,jsx}

📄 CodeRabbit Inference Engine (.cursor/rules/frontend_development.mdc)

Use Lucide React for icons in the frontend.

Files:

  • src/frontend/src/icons/lazyIconImports.ts
  • src/frontend/src/icons/eagerIconImports.ts
  • src/frontend/src/icons/VLMRun/index.tsx
src/frontend/src/icons/lazyIconImports.ts

📄 CodeRabbit Inference Engine (.cursor/rules/icons.mdc)

Add your icon to the lazyIconsMapping object in src/frontend/src/icons/lazyIconImports.ts with a key that matches the backend icon string exactly.

Files:

  • src/frontend/src/icons/lazyIconImports.ts
src/frontend/src/icons/*/*.@(js|jsx|ts|tsx)

📄 CodeRabbit Inference Engine (.cursor/rules/icons.mdc)

Create a new directory for your icon in src/frontend/src/icons/YourIconName/ and add your SVG as a React component (e.g., YourIconName.jsx). The SVG component must use the isDark prop to support both light and dark mode.

Files:

  • src/frontend/src/icons/VLMRun/index.tsx
src/frontend/src/icons/*/index.tsx

📄 CodeRabbit Inference Engine (.cursor/rules/icons.mdc)

Create an index.tsx in your icon directory that exports your icon using forwardRef and passes the isDark prop.

Files:

  • src/frontend/src/icons/VLMRun/index.tsx
src/backend/tests/unit/components/**/*.py

📄 CodeRabbit Inference Engine (.cursor/rules/backend_development.mdc)

src/backend/tests/unit/components/**/*.py: Mirror the component directory structure for unit tests in src/backend/tests/unit/components/
Use ComponentTestBaseWithClient or ComponentTestBaseWithoutClient as base classes for component unit tests
Provide file_names_mapping for backward compatibility in component tests
Create comprehensive unit tests for all new components

Files:

  • src/backend/tests/unit/components/vlmrun/test_vlmrun_transcription.py
src/backend/tests/unit/**/*.py

📄 CodeRabbit Inference Engine (.cursor/rules/backend_development.mdc)

Test component integration within flows using create_flow, build_flow, and get_build_events utilities

Files:

  • src/backend/tests/unit/components/vlmrun/test_vlmrun_transcription.py
src/backend/tests/**/*.py

📄 CodeRabbit Inference Engine (.cursor/rules/testing.mdc)

src/backend/tests/**/*.py: Unit tests for backend code must be located in the 'src/backend/tests/' directory, with component tests organized by component subdirectory under 'src/backend/tests/unit/components/'.
Test files should use the same filename as the component under test, with an appropriate test prefix or suffix (e.g., 'my_component.py' → 'test_my_component.py').
Use the 'client' fixture (an async httpx.AsyncClient) for API tests in backend Python tests, as defined in 'src/backend/tests/conftest.py'.
When writing component tests, inherit from the appropriate base class in 'src/backend/tests/base.py' (ComponentTestBase, ComponentTestBaseWithClient, or ComponentTestBaseWithoutClient) and provide the required fixtures: 'component_class', 'default_kwargs', and 'file_names_mapping'.
Each test in backend Python test files should have a clear docstring explaining its purpose, and complex setups or mocks should be well-commented.
Test both sync and async code paths in backend Python tests, using '@pytest.mark.asyncio' for async tests.
Mock external dependencies appropriately in backend Python tests to isolate unit tests from external services.
Test error handling and edge cases in backend Python tests, including using 'pytest.raises' and asserting error messages.
Validate input/output behavior and test component initialization and configuration in backend Python tests.
Use the 'no_blockbuster' pytest marker to skip the blockbuster plugin in tests when necessary.
Be aware of ContextVar propagation in async tests; test both direct event loop execution and 'asyncio.to_thread' scenarios to ensure proper context isolation.
Test error handling by mocking internal functions using monkeypatch in backend Python tests.
Test resource cleanup in backend Python tests by using fixtures that ensure proper initialization and cleanup of resources.
Test timeout and performance constraints in backend Python tests using 'asyncio.wait_for' and timing assertions.
Test Langflow's Messag...

Files:

  • src/backend/tests/unit/components/vlmrun/test_vlmrun_transcription.py
🧠 Learnings (7)
📚 Learning: 2025-07-28T15:56:47.865Z
Learnt from: CR
PR: langflow-ai/langflow#0
File: .cursor/rules/icons.mdc:0-0
Timestamp: 2025-07-28T15:56:47.865Z
Learning: Applies to src/frontend/src/icons/lazyIconImports.ts : Add your icon to the `lazyIconsMapping` object in `src/frontend/src/icons/lazyIconImports.ts` with a key that matches the backend icon string exactly.

Applied to files:

  • src/frontend/src/icons/lazyIconImports.ts
  • src/frontend/src/icons/eagerIconImports.ts
📚 Learning: 2025-07-18T18:27:12.609Z
Learnt from: CR
PR: langflow-ai/langflow#0
File: .cursor/rules/frontend_development.mdc:0-0
Timestamp: 2025-07-18T18:27:12.609Z
Learning: Applies to src/frontend/src/icons/**/*.{ts,tsx,js,jsx} : Use Lucide React for icons in the frontend.

Applied to files:

  • src/frontend/src/icons/eagerIconImports.ts
  • src/frontend/src/icons/VLMRun/index.tsx
📚 Learning: 2025-07-28T15:56:47.865Z
Learnt from: CR
PR: langflow-ai/langflow#0
File: .cursor/rules/icons.mdc:0-0
Timestamp: 2025-07-28T15:56:47.865Z
Learning: Applies to src/frontend/src/icons/*/*.@(js|jsx|ts|tsx) : Create a new directory for your icon in `src/frontend/src/icons/YourIconName/` and add your SVG as a React component (e.g., `YourIconName.jsx`). The SVG component must use the `isDark` prop to support both light and dark mode.

Applied to files:

  • src/frontend/src/icons/eagerIconImports.ts
  • src/frontend/src/icons/VLMRun/index.tsx
📚 Learning: 2025-07-28T15:56:47.865Z
Learnt from: CR
PR: langflow-ai/langflow#0
File: .cursor/rules/icons.mdc:0-0
Timestamp: 2025-07-28T15:56:47.865Z
Learning: Applies to src/frontend/src/icons/*/index.tsx : Create an `index.tsx` in your icon directory that exports your icon using `forwardRef` and passes the `isDark` prop.

Applied to files:

  • src/frontend/src/icons/VLMRun/index.tsx
📚 Learning: 2025-06-16T11:14:04.200Z
Learnt from: dolfim-ibm
PR: langflow-ai/langflow#8394
File: src/frontend/src/icons/Docling/index.tsx:4-6
Timestamp: 2025-06-16T11:14:04.200Z
Learning: The Langflow codebase consistently uses `React.PropsWithChildren<{}>` as the prop type for all icon components using forwardRef, rather than `React.SVGProps<SVGSVGElement>`. This is an established pattern across hundreds of icon files in src/frontend/src/icons/.

Applied to files:

  • src/frontend/src/icons/VLMRun/index.tsx
📚 Learning: 2025-06-23T12:46:52.420Z
Learnt from: CR
PR: langflow-ai/langflow#0
File: .cursor/rules/icons.mdc:0-0
Timestamp: 2025-06-23T12:46:52.420Z
Learning: Export custom icon components in React using React.forwardRef to ensure proper ref forwarding and compatibility with parent components.

Applied to files:

  • src/frontend/src/icons/VLMRun/index.tsx
📚 Learning: 2025-07-18T18:25:54.486Z
Learnt from: CR
PR: langflow-ai/langflow#0
File: .cursor/rules/backend_development.mdc:0-0
Timestamp: 2025-07-18T18:25:54.486Z
Learning: Applies to src/backend/tests/unit/components/**/*.py : Create comprehensive unit tests for all new components

Applied to files:

  • src/backend/tests/unit/components/vlmrun/test_vlmrun_transcription.py
🧬 Code Graph Analysis (4)
src/backend/base/langflow/components/vlmrun/__init__.py (1)
src/backend/base/langflow/components/vlmrun/vlmrun_transcription.py (1)
  • VLMRunTranscription (16-229)
src/frontend/src/icons/eagerIconImports.ts (1)
src/frontend/src/icons/VLMRun/index.tsx (1)
  • VLMRunIcon (4-52)
src/backend/base/langflow/components/vlmrun/vlmrun_transcription.py (3)
src/backend/base/langflow/inputs/inputs.py (4)
  • DropdownInput (467-491)
  • FileInput (612-622)
  • MessageTextInput (205-256)
  • SecretStrInput (285-340)
src/backend/base/langflow/template/field/base.py (1)
  • Output (181-257)
src/backend/base/langflow/schema/data.py (1)
  • Data (23-277)
src/backend/tests/unit/components/vlmrun/test_vlmrun_transcription.py (4)
src/backend/base/langflow/components/vlmrun/vlmrun_transcription.py (2)
  • VLMRunTranscription (16-229)
  • process_media (78-229)
src/backend/base/langflow/schema/data.py (1)
  • Data (23-277)
src/backend/tests/base.py (1)
  • ComponentTestBaseWithoutClient (163-164)
src/backend/base/langflow/custom/custom_component/component.py (1)
  • to_frontend_node (912-964)
🔇 Additional comments (7)
src/frontend/src/utils/styleUtils.ts (1)

303-304: VLM Run bundle naming alignment confirmed

I’ve verified that the backend components (src/backend/base/langflow/components/vlmrun/...) use the directory and category name “vlmrun” with icon "VLMRun", and that the frontend entries in src/frontend/src/utils/styleUtils.ts (lines 303–304), as well as the eager and lazy icon imports, consistently use name: "vlmrun" and icon: "VLMRun". No mismatches found—everything is aligned.

src/frontend/src/icons/eagerIconImports.ts (2)

109-110: Import added for VLMRun icon is correct.

The import path matches the icon’s index.tsx and the named export aligns with lazy mapping.


226-227: Eager registry entry added; ensure backend icon string matches exactly ("VLMRun").

Key casing matches SIDEBAR_BUNDLES and backend metadata. No further changes needed.

src/frontend/src/icons/lazyIconImports.ts (1)

283-286: Lazy icon mapping looks good and consistent with the module’s named export.

VLMRun is correctly mapped to mod.VLMRunIcon, matching the eager map and sidebar usage.

src/backend/base/langflow/components/vlmrun/__init__.py (1)

1-3: Clean export surface; matches component module structure

Re-exporting VLMRunTranscription and declaring all keeps the package boundary tidy and importable via langflow.components.vlmrun. No issues spotted.

src/backend/tests/unit/components/vlmrun/test_vlmrun_transcription.py (1)

75-85: Good negative-path validation

The “no input” case asserts both the Data payload and component.status. This covers the main guard clause. Looks good.

src/backend/base/langflow/components/vlmrun/vlmrun_transcription.py (1)

20-21: Icon string matches the expected mapping

icon = "VLMRun" is case-sensitive and aligns with the tests. Ensure the frontend bundle exports this icon under the exact key.

If needed, run the verification script in the package init review comment.

Comment on lines +1 to +3
from .vlmrun_transcription import VLMRunTranscription

__all__ = ["VLMRunTranscription"]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Verify upstream aggregation (init.py) and icon mapping are wired

Ensure the parent components/init.py includes this subpackage in alphabetical order (if that file enumerates submodules) and that the frontend icon registry contains a "VLMRun" mapping, since the backend sets icon = "VLMRun".

Run this script to confirm:


🏁 Script executed:

#!/bin/bash
# 1) Check parent components/__init__.py exposes vlmrun or VLMRunTranscription
fd -a 'components/__init__.py' | while read -r f; do
  echo "== $f"; rg -nC2 -e '\bvlmrun\b' -e 'VLMRunTranscription' "$f" || true
done

# 2) Confirm the icon "VLMRun" exists in the frontend codebase
rg -nCI2 --glob '!**/node_modules/**' --glob '!**/.venv/**' '\bVLMRun\b' src || true

Length of output: 203


🏁 Script executed:

#!/bin/bash

# 1) Locate and inspect the parent components/__init__.py under src/backend/base/langflow/components
find src/backend/base/langflow/components -maxdepth 1 -type f -name "__init__.py" | while read -r file; do
  echo "== Checking $file"
  grep -nE "vlmrun|VLMRunTranscription" "$file" || echo "  → No matches in $file"
done

# 2) Search the frontend (and overall src/) for any references to the "VLMRun" icon mapping
rg -nC2 --glob '!**/node_modules/**' --glob '!**/.venv/**' -e 'VLMRun' src || echo "→ No 'VLMRun' references found in src/"

Length of output: 6972


Aggregate the VLMRun component in the backend package __init__.py

It looks like the frontend icon mapping for "VLMRun" is already correctly wired (found in src/frontend/src/utils/styleUtils.ts and both eager and lazy icon imports). However, the backend’s parent package index isn’t exposing the new vlmrun subpackage, so the component won’t be discovered at runtime.

• File needing update:
src/backend/base/langflow/components/__init__.py
– Add an import for the VLMRun subpackage in alphabetical order.
– Include its export in the __all__ list.

Suggested diff (adjust spacing/ordering to match existing style):

 from .your_other_component import OtherComponent
+from .vlmrun import VLMRunTranscription

 __all__ = [
     "OtherComponent",
+    "VLMRunTranscription",
 ]

Please add this so that langflow.components.vlmrun and its VLMRunTranscription class are properly aggregated.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
from .vlmrun_transcription import VLMRunTranscription
__all__ = ["VLMRunTranscription"]
# File: src/backend/base/langflow/components/__init__.py
from .your_other_component import OtherComponent
from .vlmrun import VLMRunTranscription
__all__ = [
"OtherComponent",
"VLMRunTranscription",
]
🤖 Prompt for AI Agents
In src/backend/base/langflow/components/__init__.py (around where other
subpackages are imported), the new vlmrun subpackage isn't imported or exported
so langflow.components.vlmrun and VLMRunTranscription won't be discovered; add
an import line for the vlmrun package (e.g., from . import vlmrun) in
alphabetical order with the other subpackage imports and include "vlmrun" in the
__all__ list so the package aggregates the submodule and its VLMRunTranscription
export.

Comment on lines +89 to +97
# Mock the vlmrun module to not exist
def mock_import(name, *args):
if name == "vlmrun.client":
msg = "No module named 'vlmrun'"
raise ImportError(msg)
return __import__(name, *args)

monkeypatch.setattr("builtins.__import__", mock_import)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Fix recursive import patching that will cause a recursion error

The custom mock_import calls import after monkeypatching builtins.import, which becomes the mock itself, leading to infinite recursion. Capture the original importer before patching and call that.

Apply this diff:

-        # Mock the vlmrun module to not exist
-        def mock_import(name, *args):
-            if name == "vlmrun.client":
-                msg = "No module named 'vlmrun'"
-                raise ImportError(msg)
-            return __import__(name, *args)
-
-        monkeypatch.setattr("builtins.__import__", mock_import)
+        # Mock the vlmrun module to not exist without recursive imports
+        import builtins
+        original_import = builtins.__import__
+
+        def mock_import(name, globals=None, locals=None, fromlist=(), level=0):
+            if name == "vlmrun.client" or (name == "vlmrun" and "client" in (fromlist or ())):
+                raise ImportError("No module named 'vlmrun'")
+            return original_import(name, globals, locals, fromlist, level)
+
+        monkeypatch.setattr(builtins, "__import__", mock_import)
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
# Mock the vlmrun module to not exist
def mock_import(name, *args):
if name == "vlmrun.client":
msg = "No module named 'vlmrun'"
raise ImportError(msg)
return __import__(name, *args)
monkeypatch.setattr("builtins.__import__", mock_import)
# Mock the vlmrun module to not exist without recursive imports
import builtins
original_import = builtins.__import__
def mock_import(name, globals=None, locals=None, fromlist=(), level=0):
if name == "vlmrun.client" or (name == "vlmrun" and "client" in (fromlist or ())):
raise ImportError("No module named 'vlmrun'")
return original_import(name, globals, locals, fromlist, level)
monkeypatch.setattr(builtins, "__import__", mock_import)
🤖 Prompt for AI Agents
In src/backend/tests/unit/components/vlmrun/test_vlmrun_transcription.py around
lines 89 to 97, the mock_import currently calls __import__ after monkeypatching
builtins.__import__, causing infinite recursion; capture the original
builtins.__import__ into a local variable before calling monkeypatch.setattr,
and have mock_import call that saved original importer when delegating to import
modules other than "vlmrun.client"; then patch builtins.__import__ with
mock_import as before.

Comment on lines +1 to +52
import type React from "react";
import { forwardRef } from "react";

export const VLMRunIcon = forwardRef<
SVGSVGElement,
React.PropsWithChildren<{
className?: string;
}>
>(({ className, ...props }, ref) => {
return (
<svg
ref={ref}
width="171"
height="130"
viewBox="0 0 171 130"
fill="none"
xmlns="http://www.w3.org/2000/svg"
className={className}
{...props}
>
<path
d="M10.4983 16.086C10.4983 7.20196 17.7003 0 26.5843 0H47.581V10.724H26.5843C23.623 10.724 21.2223 13.1247 21.2223 16.086V58.1921C21.2223 61.1535 18.8217 63.5541 15.8603 63.5541L15.8603 66.1222C18.8217 66.1222 21.2223 68.5229 21.2223 71.4842V113.59C21.2223 116.552 23.623 118.952 26.5843 118.952H47.581V129.676H26.5843C17.7003 129.676 10.4983 122.474 10.4983 113.59V70.2002H0V59.4762H10.4983V16.086Z"
fill="#1E5ACA"
/>
<path
d="M44.3593 60.8161L32.4557 27.7432H42.1716L50.1503 51.5506L58.0003 27.7432H67.5232L55.6195 60.8161H44.3593Z"
fill="#1E5ACA"
/>
<path d="M69.26 60.8161V15.7752H78.3325V60.8161H69.26Z" fill="#1E5ACA" />
<path
d="M122.94 27.3572C129.76 27.3572 135.165 31.2178 135.165 41.8346V60.8161H126.092V42.3493C126.092 37.3949 124.291 34.8211 120.366 34.8211C116.441 34.8211 113.803 37.7166 113.803 42.7354V60.8161H104.73V42.3493C104.73 37.3949 102.929 34.8211 98.9392 34.8211C95.0142 34.8211 92.4404 37.8453 92.4404 42.7998V60.8161H83.3679V27.7432H91.2822L92.1187 31.8613C94.1134 29.2875 97.0089 27.4215 101.835 27.3572C105.888 27.2928 109.685 28.7727 111.744 32.9551C114.06 29.4162 117.921 27.3572 122.94 27.3572Z"
fill="#1E5ACA"
/>
<path
d="M58.3526 73.3203H59.8969V81.5564H56.2936C50.8887 81.5564 48.8297 85.1596 48.8297 90.1785V106.393H39.7572V73.3203H47.9932L48.8297 78.2748C50.6313 75.315 53.2051 73.3203 58.3526 73.3203Z"
fill="#1E5ACA"
/>
<path
d="M84.7322 89.8567V73.3203H93.8047V106.393H85.7617L84.9252 102.404C82.8019 104.978 79.9064 106.779 75.2093 106.779C68.3888 106.779 62.0831 103.369 62.0831 91.6584V73.3203H71.1556V90.4358C71.1556 96.3555 73.0859 99.251 77.59 99.251C82.0941 99.251 84.7322 95.9694 84.7322 89.8567Z"
fill="#1E5ACA"
/>
<path
d="M117.807 72.9342C124.949 72.9342 131.062 76.7305 131.062 88.0551V106.393H121.989V88.9559C121.989 83.358 119.866 80.3982 115.298 80.3982C110.601 80.3982 107.898 83.744 107.898 89.4707V106.393H98.8256V73.3203H106.74L107.576 77.4383C109.7 74.8646 112.724 72.9342 117.807 72.9342Z"
fill="#1E5ACA"
/>
<path
d="M159.732 16.086C159.732 7.20196 152.53 0 143.646 0H122.649V10.724H143.646C146.607 10.724 149.008 13.1247 149.008 16.086V58.1921C149.008 61.1535 151.408 63.5541 154.37 63.5541V66.1222C151.408 66.1222 149.008 68.5229 149.008 71.4842V113.59C149.008 116.552 146.607 118.952 143.646 118.952H122.649V129.676H143.646C152.53 129.676 159.732 122.474 159.732 113.59V70.2002H170.23V59.4762H159.732V16.086Z"
fill="#1E5ACA"
/>
</svg>
);
});
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Support dark mode via isDark and prefer scalable sizing (1em) over fixed px.

Current SVG hardcodes fill color and fixed width/height; this breaks theme adaptability and makes sizing inconsistent with other icons. Follow the repository’s icon pattern: accept isDark prop, use it to compute fill (or use currentColor), and use 1em sizing to respect parent text size or Tailwind classes.

Apply this diff:

-import type React from "react";
-import { forwardRef } from "react";
+import type React from "react";
+import { forwardRef } from "react";

 export const VLMRunIcon = forwardRef<
   SVGSVGElement,
-  React.PropsWithChildren<{
-    className?: string;
-  }>
->(({ className, ...props }, ref) => {
+  React.PropsWithChildren<{ className?: string; isDark?: boolean }>
+>(({ className, isDark, ...props }, ref) => {
+  // Prefer theming via currentColor; if brand color must be enforced, toggle here.
+  const fillColor = isDark ? "currentColor" : "currentColor";
   return (
     <svg
       ref={ref}
-      width="171"
-      height="130"
+      width="1em"
+      height="1em"
       viewBox="0 0 171 130"
-      fill="none"
+      fill="none"
       xmlns="http://www.w3.org/2000/svg"
-      className={className}
+      className={className}
+      aria-hidden="true"
+      focusable="false"
       {...props}
     >
-      <path
+      <path
         d="M10.4983 16.086C10.4983 7.20196 17.7003 0 26.5843 0H47.581V10.724H26.5843C23.623 10.724 21.2223 13.1247 21.2223 16.086V58.1921C21.2223 61.1535 18.8217 63.5541 15.8603 63.5541L15.8603 66.1222C18.8217 66.1222 21.2223 68.5229 21.2223 71.4842V113.59C21.2223 116.552 23.623 118.952 26.5843 118.952H47.581V129.676H26.5843C17.7003 129.676 10.4983 122.474 10.4983 113.59V70.2002H0V59.4762H10.4983V16.086Z"
-        fill="#1E5ACA"
+        fill={fillColor}
       />
       <path
         d="M44.3593 60.8161L32.4557 27.7432H42.1716L50.1503 51.5506L58.0003 27.7432H67.5232L55.6195 60.8161H44.3593Z"
-        fill="#1E5ACA"
+        fill={fillColor}
       />
-      <path d="M69.26 60.8161V15.7752H78.3325V60.8161H69.26Z" fill="#1E5ACA" />
+      <path d="M69.26 60.8161V15.7752H78.3325V60.8161H69.26Z" fill={fillColor} />
       <path
         d="M122.94 27.3572C129.76 27.3572 135.165 31.2178 135.165 41.8346V60.8161H126.092V42.3493C126.092 37.3949 124.291 34.8211 120.366 34.8211C116.441 34.8211 113.803 37.7166 113.803 42.7354V60.8161H104.73V42.3493C104.73 37.3949 102.929 34.8211 98.9392 34.8211C95.0142 34.8211 92.4404 37.8453 92.4404 42.7998V60.8161H83.3679V27.7432H91.2822L92.1187 31.8613C94.1134 29.2875 97.0089 27.4215 101.835 27.3572C105.888 27.2928 109.685 28.7727 111.744 32.9551C114.06 29.4162 117.921 27.3572 122.94 27.3572Z"
-        fill="#1E5ACA"
+        fill={fillColor}
       />
       <path
         d="M58.3526 73.3203H59.8969V81.5564H56.2936C50.8887 81.5564 48.8297 85.1596 48.8297 90.1785V106.393H39.7572V73.3203H47.9932L48.8297 78.2748C50.6313 75.315 53.2051 73.3203 58.3526 73.3203Z"
-        fill="#1E5ACA"
+        fill={fillColor}
       />
       <path
         d="M84.7322 89.8567V73.3203H93.8047V106.393H85.7617L84.9252 102.404C82.8019 104.978 79.9064 106.779 75.2093 106.779C68.3888 106.779 62.0831 103.369 62.0831 91.6584V73.3203H71.1556V90.4358C71.1556 96.3555 73.0859 99.251 77.59 99.251C82.0941 99.251 84.7322 95.9694 84.7322 89.8567Z"
-        fill="#1E5ACA"
+        fill={fillColor}
       />
       <path
         d="M117.807 72.9342C124.949 72.9342 131.062 76.7305 131.062 88.0551V106.393H121.989V88.9559C121.989 83.358 119.866 80.3982 115.298 80.3982C110.601 80.3982 107.898 83.744 107.898 89.4707V106.393H98.8256V73.3203H106.74L107.576 77.4383C109.7 74.8646 112.724 72.9342 117.807 72.9342Z"
-        fill="#1E5ACA"
+        fill={fillColor}
       />
       <path
         d="M159.732 16.086C159.732 7.20196 152.53 0 143.646 0H122.649V10.724H143.646C146.607 10.724 149.008 13.1247 149.008 16.086V58.1921C149.008 61.1535 151.408 63.5541 154.37 63.5541V66.1222C151.408 66.1222 149.008 68.5229 149.008 71.4842V113.59C149.008 116.552 146.607 118.952 143.646 118.952H122.649V129.676H143.646C152.53 129.676 159.732 122.474 159.732 113.59V70.2002H170.23V59.4762H159.732V16.086Z"
-        fill="#1E5ACA"
+        fill={fillColor}
       />
     </svg>
   );
 });

If the brand color must be preserved, replace currentColor with your desired light/dark hex values.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
import type React from "react";
import { forwardRef } from "react";
export const VLMRunIcon = forwardRef<
SVGSVGElement,
React.PropsWithChildren<{
className?: string;
}>
>(({ className, ...props }, ref) => {
return (
<svg
ref={ref}
width="171"
height="130"
viewBox="0 0 171 130"
fill="none"
xmlns="http://www.w3.org/2000/svg"
className={className}
{...props}
>
<path
d="M10.4983 16.086C10.4983 7.20196 17.7003 0 26.5843 0H47.581V10.724H26.5843C23.623 10.724 21.2223 13.1247 21.2223 16.086V58.1921C21.2223 61.1535 18.8217 63.5541 15.8603 63.5541L15.8603 66.1222C18.8217 66.1222 21.2223 68.5229 21.2223 71.4842V113.59C21.2223 116.552 23.623 118.952 26.5843 118.952H47.581V129.676H26.5843C17.7003 129.676 10.4983 122.474 10.4983 113.59V70.2002H0V59.4762H10.4983V16.086Z"
fill="#1E5ACA"
/>
<path
d="M44.3593 60.8161L32.4557 27.7432H42.1716L50.1503 51.5506L58.0003 27.7432H67.5232L55.6195 60.8161H44.3593Z"
fill="#1E5ACA"
/>
<path d="M69.26 60.8161V15.7752H78.3325V60.8161H69.26Z" fill="#1E5ACA" />
<path
d="M122.94 27.3572C129.76 27.3572 135.165 31.2178 135.165 41.8346V60.8161H126.092V42.3493C126.092 37.3949 124.291 34.8211 120.366 34.8211C116.441 34.8211 113.803 37.7166 113.803 42.7354V60.8161H104.73V42.3493C104.73 37.3949 102.929 34.8211 98.9392 34.8211C95.0142 34.8211 92.4404 37.8453 92.4404 42.7998V60.8161H83.3679V27.7432H91.2822L92.1187 31.8613C94.1134 29.2875 97.0089 27.4215 101.835 27.3572C105.888 27.2928 109.685 28.7727 111.744 32.9551C114.06 29.4162 117.921 27.3572 122.94 27.3572Z"
fill="#1E5ACA"
/>
<path
d="M58.3526 73.3203H59.8969V81.5564H56.2936C50.8887 81.5564 48.8297 85.1596 48.8297 90.1785V106.393H39.7572V73.3203H47.9932L48.8297 78.2748C50.6313 75.315 53.2051 73.3203 58.3526 73.3203Z"
fill="#1E5ACA"
/>
<path
d="M84.7322 89.8567V73.3203H93.8047V106.393H85.7617L84.9252 102.404C82.8019 104.978 79.9064 106.779 75.2093 106.779C68.3888 106.779 62.0831 103.369 62.0831 91.6584V73.3203H71.1556V90.4358C71.1556 96.3555 73.0859 99.251 77.59 99.251C82.0941 99.251 84.7322 95.9694 84.7322 89.8567Z"
fill="#1E5ACA"
/>
<path
d="M117.807 72.9342C124.949 72.9342 131.062 76.7305 131.062 88.0551V106.393H121.989V88.9559C121.989 83.358 119.866 80.3982 115.298 80.3982C110.601 80.3982 107.898 83.744 107.898 89.4707V106.393H98.8256V73.3203H106.74L107.576 77.4383C109.7 74.8646 112.724 72.9342 117.807 72.9342Z"
fill="#1E5ACA"
/>
<path
d="M159.732 16.086C159.732 7.20196 152.53 0 143.646 0H122.649V10.724H143.646C146.607 10.724 149.008 13.1247 149.008 16.086V58.1921C149.008 61.1535 151.408 63.5541 154.37 63.5541V66.1222C151.408 66.1222 149.008 68.5229 149.008 71.4842V113.59C149.008 116.552 146.607 118.952 143.646 118.952H122.649V129.676H143.646C152.53 129.676 159.732 122.474 159.732 113.59V70.2002H170.23V59.4762H159.732V16.086Z"
fill="#1E5ACA"
/>
</svg>
);
});
import type React from "react";
import { forwardRef } from "react";
export const VLMRunIcon = forwardRef<
SVGSVGElement,
React.PropsWithChildren<{ className?: string; isDark?: boolean }>
>(({ className, isDark, ...props }, ref) => {
// Prefer theming via currentColor; if brand color must be enforced, toggle here.
const fillColor = isDark ? "currentColor" : "currentColor";
return (
<svg
ref={ref}
width="1em"
height="1em"
viewBox="0 0 171 130"
fill="none"
xmlns="http://www.w3.org/2000/svg"
className={className}
aria-hidden="true"
focusable="false"
{...props}
>
<path
d="M10.4983 16.086C10.4983 7.20196 17.7003 0 26.5843 0H47.581V10.724H26.5843C23.623 10.724 21.2223 13.1247 21.2223 16.086V58.1921C21.2223 61.1535 18.8217 63.5541 15.8603 63.5541L15.8603 66.1222C18.8217 66.1222 21.2223 68.5229 21.2223 71.4842V113.59C21.2223 116.552 23.623 118.952 26.5843 118.952H47.581V129.676H26.5843C17.7003 129.676 10.4983 122.474 10.4983 113.59V70.2002H0V59.4762H10.4983V16.086Z"
fill={fillColor}
/>
<path
d="M44.3593 60.8161L32.4557 27.7432H42.1716L50.1503 51.5506L58.0003 27.7432H67.5232L55.6195 60.8161H44.3593Z"
fill={fillColor}
/>
<path
d="M69.26 60.8161V15.7752H78.3325V60.8161H69.26Z"
fill={fillColor}
/>
<path
d="M122.94 27.3572C129.76 27.3572 135.165 31.2178 135.165 41.8346V60.8161H126.092V42.3493C126.092 37.3949 124.291 34.8211 120.366 34.8211C116.441 34.8211 113.803 37.7166 113.803 42.7354V60.8161H104.73V42.3493C104.73 37.3949 102.929 34.8211 98.9392 34.8211C95.0142 34.8211 92.4404 37.8453 92.4404 42.7998V60.8161H83.3679V27.7432H91.2822L92.1187 31.8613C94.1134 29.2875 97.0089 27.4215 101.835 27.3572C105.888 27.2928 109.685 28.7727 111.744 32.9551C114.06 29.4162 117.921 27.3572 122.94 27.3572Z"
fill={fillColor}
/>
<path
d="M58.3526 73.3203H59.8969V81.5564H56.2936C50.8887 81.5564 48.8297 85.1596 48.8297 90.1785V106.393H39.7572V73.3203H47.9932L48.8297 78.2748C50.6313 75.315 53.2051 73.3203 58.3526 73.3203Z"
fill={fillColor}
/>
<path
d="M84.7322 89.8567V73.3203H93.8047V106.393H85.7617L84.9252 102.404C82.8019 104.978 79.9064 106.779 75.2093 106.779C68.3888 106.779 62.0831 103.369 62.0831 91.6584V73.3203H71.1556V90.4358C71.1556 96.3555 73.0859 99.251 77.59 99.251C82.0941 99.251 84.7322 95.9694 84.7322 89.8567Z"
fill={fillColor}
/>
<path
d="M117.807 72.9342C124.949 72.9342 131.062 76.7305 131.062 88.0551V106.393H121.989V88.9559C121.989 83.358 119.866 80.3982 115.298 80.3982C110.601 80.3982 107.898 83.744 107.898 89.4707V106.393H98.8256V73.3203H106.74L107.576 77.4383C109.7 74.8646 112.724 72.9342 117.807 72.9342Z"
fill={fillColor}
/>
<path
d="M159.732 16.086C159.732 7.20196 152.53 0 143.646 0H122.649V10.724H143.646C146.607 10.724 149.008 13.1247 149.008 16.086V58.1921C149.008 61.1535 151.408 63.5541 154.37 63.5541V66.1222C151.408 66.1222 149.008 68.5229 149.008 71.4842V113.59C149.008 116.552 146.607 118.952 143.646 118.952H122.649V129.676H143.646C152.53 129.676 159.732 122.474 159.732 113.59V70.2002H170.23V59.4762H159.732V16.086Z"
fill={fillColor}
/>
</svg>
);
});

@github-actions github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Aug 20, 2025
@wbrennan899 wbrennan899 force-pushed the vlm-run-video-audio-integration branch from a0bced0 to 2ffe4e5 Compare August 20, 2025 22:05
@github-actions github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Aug 20, 2025

def _format_output(self, all_results: list) -> dict:
"""Format the output based on number of results."""
if len(all_results) == 1:
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should not have such hardcoded way to format results.

def _wait_for_response(self, client, response):
"""Wait for batch processing to complete if needed."""
if hasattr(response, "id"):
return client.predictions.wait(response.id, timeout=600)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we expose timeout for client.predictions.wait(..., timeout=600) as an advanced input (e.g., timeout_seconds, default 600)? This avoids hard-coding and helps long videos.

}

# Add source-specific field
if source_name.startswith("http"):
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use a more robust URL check than source_name.startswith("http") (e.g., urllib.parse.urlparse(source_name).scheme), supporting schemes like https, s3, gs.

"media_type": self.media_type,
"duration": response_data.get("metadata", {}).get("duration", 0),
},
"usage": response.usage.__dict__ if hasattr(response, "usage") else None,
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid response.usage.dict—prefer a stable method if provided (e.g., model_dump() or asdict()), or explicitly map known fields to a dict to avoid coupling to object internals.

def _generate_media_response(self, client, media_source):
"""Generate response for audio or video media."""
if self.media_type == "audio":
if isinstance(media_source, Path):
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make domain configurable ("audio.transcription" / "video.transcription") as an advanced input, in case VLM Run adds new domains or variants (e.g., diarization).

from tests.base import ComponentTestBaseWithoutClient


class TestVLMRunTranscription(ComponentTestBaseWithoutClient):
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Single audio file with mocked client returning segments (assert transcription, metadata.duration, status, prediction_id).

Video flow (ensure video content is included, and optional audio in brackets is appended as implemented).

@github-actions github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Aug 22, 2025
@erichare erichare self-requested a review August 27, 2025 14:54
@wbrennan899 wbrennan899 force-pushed the vlm-run-video-audio-integration branch from 9a697d4 to 02d9d10 Compare August 27, 2025 21:40
@github-actions github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Aug 27, 2025
Copy link
Copy Markdown
Collaborator

@erichare erichare left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a fantastic addition and a very well constructred PR. I'm going to approve it, but i would ask for a couple things:

  1. Could you please address (or at least comment on) all outstanding comments in the PR?
  2. If there are some easy ways to add basic unit tests, that would be a great addition - with that said, i understand this is challenging for a service that requires an external API key. This would be EXTREMELY basic.
  3. Keep an eye on any mypy / ruff / pytest checks that may need to be addressed before merge.

Thanks so much.

@erichare erichare added the lgtm This PR has been approved by a maintainer label Sep 5, 2025
@erichare
Copy link
Copy Markdown
Collaborator

erichare commented Sep 5, 2025

CC @ogabrielluiz this seems like a very solid bundle for the next release, would love your thoughts. But it's well designed and isolated, with the proper bundle semantics as far as I can tell.

@erichare
Copy link
Copy Markdown
Collaborator

@wbrennan899

Can you update the init to look like:

from __future__ import annotations

from typing import TYPE_CHECKING, Any

from langflow.components._importing import import_mod

if TYPE_CHECKING:
    from .vlmrun_transcription import VLMRunTranscription

_dynamic_imports = {
    "VLMRunTranscription": "vlmrun_transcription",
}

__all__ = [
    "VLMRunTranscription",
]


def __getattr__(attr_name: str) -> Any:
    """Lazily import Vectara components on attribute access."""
    if attr_name not in _dynamic_imports:
        msg = f"module '{__name__}' has no attribute '{attr_name}'"
        raise AttributeError(msg)
    try:
        result = import_mod(attr_name, _dynamic_imports[attr_name], __spec__.parent)
    except (ModuleNotFoundError, ImportError, AttributeError) as e:
        msg = f"Could not import '{attr_name}' from '{__name__}': {e}"
        raise AttributeError(msg) from e
    globals()[attr_name] = result
    return result


def __dir__() -> list[str]:
    return list(__all__)

(Double check my work)

That should fix the import error

@wbrennan899 wbrennan899 force-pushed the vlm-run-video-audio-integration branch from 02d9d10 to 35160e2 Compare September 11, 2025 18:03
@github-actions github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Sep 11, 2025
@erichare
Copy link
Copy Markdown
Collaborator

@erichare what is the import error that you are addressing?

@wbrennan899 If you check Group 5 of the backend tests, they're failing currently.

  src/backend/tests/unit/components/vlmrun/test_vlmrun_transcription.py:4: in <module>
      from langflow.components.vlmrun.vlmrun_transcription import VLMRunTranscription
  E   ModuleNotFoundError: No module named 'langflow.components.vlmrun'

@wbrennan899
Copy link
Copy Markdown
Contributor Author

@erichare Thanks.

As a side note, I am having issues testing locally due to a new torch dependency. I filed an issue here (#9824). I'll temporarily remove the torch dependency for now for testing.

@github-actions github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Sep 11, 2025
@github-actions github-actions Bot added the enhancement New feature or request label Oct 8, 2025
@wbrennan899
Copy link
Copy Markdown
Contributor Author

@erichare I fixed the recent uv.lock conflict that caused this to be removed from the merge queue

@erichare
Copy link
Copy Markdown
Collaborator

erichare commented Oct 8, 2025

@wbrennan899 sorry this has been such a pain. We are very close and thank you for your work. Can you do two things:

  1. Resolve latest conflicts
  2. Pull latest main

@wbrennan899 wbrennan899 force-pushed the vlm-run-video-audio-integration branch from 08238e8 to 1ad9a46 Compare October 9, 2025 00:43
@github-actions github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Oct 9, 2025
@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud Bot commented Oct 9, 2025

@wbrennan899
Copy link
Copy Markdown
Contributor Author

@erichare I resolved the latest conflicts

@erichare erichare enabled auto-merge October 9, 2025 00:55
@erichare erichare added this pull request to the merge queue Oct 9, 2025
Merged via the queue into langflow-ai:main with commit 67960cf Oct 9, 2025
116 of 118 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request lgtm This PR has been approved by a maintainer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants