feat: Add Openlayer Integration

[viniciusdsmello]

Pull Request

Summary

Adds Openlayer tracing integration to LangFlow via OpenlayerTracer service. Enables comprehensive observability for LangFlow flows with automatic component tracking, LangChain callback integration, and hierarchical trace organization.

Changes

• Added OpenlayerTracer class implementing BaseTracer interface
• Integrated Openlayer SDK for trace/step management
• Implemented automatic LangChain callback handler integration with nested step support
• Added flexible configuration via environment variables (flow-specific, JSON mapping, or default)
• Implemented type conversion for LangFlow types (Message, Data) to Openlayer-compatible formats
• Added user/session context propagation to SDK
• Added documentation

Context

Provides LangFlow users with production-grade observability through Openlayer's tracing platform. Key capabilities:

• Automatic trace capture: All components tracked with inputs/outputs/timing
• LangChain integration: Nested LLM calls automatically appear in component hierarchy
• Flexible configuration: Three-tier config system (flow-specific → JSON mapping → default)
• Zero instrumentation: Works automatically when OPENLAYER_API_KEY is set

Configuration priority:

1. OPENLAYER_PIPELINE_<FLOW_NAME> (flow-specific)
2. OPENLAYER_LANGFLOW_MAPPING (JSON mapping)
3. OPENLAYER_INFERENCE_PIPELINE_ID (default)

Example

The following Flow

Resulted on the following trace on the platform

Testing

• Manual testing with multiple flow types (simple, LLM-based, agent-based)
• Verified LangChain callback integration and nested step hierarchy
• Tested all three configuration methods
• Validated trace structure in Openlayer UI
• Confirmed zero performance impact when Openlayer is disabled

Summary by CodeRabbit

• New Features

  • Openlayer tracing integration is now available for monitoring and analyzing workflow executions.

• Documentation

  • Added comprehensive Openlayer integration guide covering prerequisites, environment configuration, setup procedures, and advanced configuration options.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

Walkthrough

This PR introduces OpenLayer integration to LangFlow's tracing system. Changes include a new OpenlayerTracer class handling trace creation and hierarchy construction, integration into the tracing service, dependency addition, documentation, and test updates.

Changes

Cohort / File(s)	Summary
Documentation docs/docs/Develop/integrations-openlayer.mdx	New documentation page describing OpenLayer integration prerequisites, environment variable configuration, flow tracing details, advanced configuration options (flow-specific pipelines, JSON mapping), configuration priority, disable instructions, and features.
Dependencies pyproject.toml	Added openlayer>=0.8.1,<1.0.0 dependency.
Tracer Implementation src/backend/base/langflow/services/tracing/openlayer.py	New OpenlayerTracer class providing initialization with flow-specific env var resolution, lazy SDK setup, trace creation via add_trace with LangFlow-to-OpenLayer type mapping, trace finalization via end_trace, hierarchy construction, LangChain callback integration, and data conversion utilities.
Service Integration src/backend/base/langflow/services/tracing/service.py	Added `_get_openlayer_tracer()` helper and `_initialize_openlayer_tracer()` method; integrated OpenlayerTracer initialization into `start_tracers()` lifecycle.
Tests src/backend/tests/unit/services/tracing/test_tracing_service.py	Patched `_get_openlayer_tracer` to return MockTracer; updated `test_start_end_tracers` to assert "openlayer" presence in trace_context.tracers.

Sequence Diagram(s)

sequenceDiagram
    participant Flow as LangFlow Flow
    participant Svc as Tracing Service
    participant OT as OpenlayerTracer
    participant SDK as OpenLayer SDK
    
    Flow->>Svc: start_tracers(trace_context)
    Svc->>OT: __init__(trace_name, trace_id, ...)
    OT->>OT: _get_config() resolves api_key & pipeline_id
    
    Note over OT: Configuration resolved,<br/>SDK setup deferred
    
    Flow->>OT: add_trace(trace_id, inputs, vertex, ...)
    OT->>OT: Convert LangFlow types to OpenLayer StepTypes
    OT->>OT: Build step hierarchy from vertex
    OT->>OT: Store in _openlayer_steps & traces
    
    Flow->>OT: end_trace(trace_id, outputs, error, ...)
    OT->>OT: Finalize step (timing, outputs, errors, logs)
    OT->>OT: Clear context
    
    Flow->>OT: end(inputs, outputs, error, ...)
    OT->>OT: setup_openlayer() initializes SDK
    OT->>OT: Build final trace hierarchy
    OT->>SDK: Stream trace to inference pipeline
    Note over SDK: Async processing
    OT->>OT: Cleanup on success/error

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

• OpenlayerTracer class (openlayer.py) — Dense logic covering trace type mapping, hierarchy construction, data conversions, and OpenLayer SDK integration; requires careful review of step relationship handling and type safety.
• Trace data flow — Verify add_trace → end_trace → end sequencing maintains correct state and context cleanup.
• Configuration resolution — Ensure flow-specific environment variables and JSON mapping are correctly prioritized.
• LangChain callback integration — Confirm AsyncOpenlayerHandler context persistence works correctly.

Possibly related PRs

• feat: Added Traceloop SDK for collecting traces and metrics from Langflow #9317 — Parallel tracer integration (Traceloop) introducing similar tracer class structure, dependency addition, and service/test integration points.

Suggested reviewers

• ogabrielluiz
• Yukiyukiyeah
• edwinjosechittilappilly

Pre-merge checks and finishing touches

Important

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

❌ Failed checks (1 error, 3 warnings)

Check name	Status	Explanation	Resolution
Test Coverage For New Implementations	❌ Error	The PR introduces 400+ lines of OpenlayerTracer implementation with no dedicated unit tests and zero openlayer-specific tests in modified test files.	Add comprehensive unit tests in test_openlayer.py covering initialization, trace creation, SDK context cleanup, type conversions, and error handling scenarios.
Docstring Coverage	⚠️ Warning	Docstring coverage is 38.10% which is insufficient. The required threshold is 80.00%.	You can run @coderabbitai generate docstrings to improve docstring coverage.
Test Quality And Coverage	⚠️ Warning	OpenlayerTracer implementation lacks comprehensive unit tests for initialization, configuration resolution, trace conversion, and error handling.	Add comprehensive unit tests covering initialization, configuration tiers, add_trace/end_trace methods, SDK context cleanup, type mapping, and error handling.
Test File Naming And Structure	⚠️ Warning	Pull request lacks dedicated test file for OpenlayerTracer class; existing tests only verify initialization in service, not the class methods, configuration logic, or error handling.	Create test_openlayer.py with comprehensive coverage of OpenlayerTracer initialization, configuration resolution, public methods, SDK context cleanup, edge cases, and LangChain callback integration.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely summarizes the main change: adding Openlayer integration to LangFlow. It is specific, directly related to the primary objective of the PR, and follows conventional commit style.
Excessive Mock Usage Warning	✅ Passed	The pull request demonstrates appropriate use of mocks in its test suite with real state tracking logic rather than empty pass statements.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (4)

src/backend/tests/unit/services/tracing/test_tracing_service.py (1)

121-123: Test coverage for Openlayer tracer registration is adequate

Patching _get_openlayer_tracer to return MockTracer and asserting "openlayer" in trace_context.tracers keeps the tests aligned with the new tracer without requiring the actual Openlayer SDK at test time. The fixture structure and the new assertion are consistent with the existing tracers; no issues from a correctness standpoint. If you ever want stricter coverage, you could also assert the presence of "opik" here for symmetry with the service initialization, but that’s optional.

Also applies to: 148-151, 177-184

src/backend/base/langflow/services/tracing/openlayer.py (3)

64-101: Consider surfacing minimal diagnostics when Openlayer setup fails

setup_openlayer defensively returns False on any missing config, ImportError, or generic Exception, which is good for not breaking LangFlow when Openlayer is misconfigured. However, this makes it hard to understand why the tracer isn’t ready (no API key vs. bad JSON vs. import failure).

A lightweight improvement would be to log at debug level before returning False for each failure path (missing required keys, import error, generic exception), so users can quickly see why OpenlayerTracer.ready is False without impacting production behavior.

This is optional, but would make diagnosing integration issues much easier.

---

285-385: Clean up unused hierarchy helpers or hook them into the step tree

Within the hierarchy utilities:

• self.component_parent_ids populated in add_trace (Lines 166–170) is never read.
• _is_tool_provider (Lines 327–331) is defined but never used.

If you don’t plan to use these in a follow‑up to build a deeper component‑level tree (e.g., nesting steps based on component_parent_ids or special‑casing tool providers), consider removing them to keep the tracer lean and easier to maintain. If you do plan to use them, adding a short comment about their intended purpose would help future readers understand why they’re there.

---

391-443: Data conversion logic is robust but can be made cheaper and slightly clearer

The recursive _convert_to_openlayer_type covers most of the important LangFlow/LangChain types (Message, Data, BaseMessage, Document, Pydantic models, tools, generators) and will generally produce sensible strings or primitives for Openlayer. A couple of optional refinements:

• Imports of Document, BaseMessage, Data, and Message happen on every call. If performance becomes a concern on large traces, you could hoist these imports to module level or cache them in a small helper to avoid repeated import overhead.
• The types.GeneratorType | types.NoneType isinstance check works, but the special‑case is not strictly needed since both are handled by the string fallback. You can simplify by letting them fall through to the final str(value) case (or explicitly check only for GeneratorType if you want to avoid accidentally stringifying other “tool‑like” objects).

These are micro‑optimizations / clarity tweaks; correctness looks fine as is.

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 79d02f2 and f463a47.

⛔ Files ignored due to path filters (1)

• uv.lock is excluded by !**/*.lock

📒 Files selected for processing (5)

• docs/docs/Develop/integrations-openlayer.mdx (1 hunks)
• pyproject.toml (1 hunks)
• src/backend/base/langflow/services/tracing/openlayer.py (1 hunks)
• src/backend/base/langflow/services/tracing/service.py (3 hunks)
• src/backend/tests/unit/services/tracing/test_tracing_service.py (2 hunks)

🧰 Additional context used

🧬 Code graph analysis (1)

src/backend/base/langflow/services/tracing/openlayer.py (4)

src/lfx/src/lfx/schema/message.py (1)

• Message (34-299)

src/backend/base/langflow/services/tracing/base.py (1)

• BaseTracer (16-71)

src/lfx/src/lfx/field_typing/constants.py (1)

• Document (43-44)

src/lfx/src/lfx/schema/data.py (1)

• Data (26-288)

🔇 Additional comments (3)

pyproject.toml (1)

140-140: Openlayer dependency wiring looks consistent with the new tracer

Adding openlayer as a core dependency matches the new OpenlayerTracer integration and avoids runtime ImportError. Please just double‑check this version range against what you validated in your environment (API surface, performance, and security advisories), since my view of upstream releases may lag yours.

docs/docs/Develop/integrations-openlayer.mdx (1)

1-139: Docs nicely mirror the implemented configuration behavior

The integration guide clearly matches the code: env var names, flow‑specific OPENLAYER_PIPELINE_<FLOW_NAME> handling, JSON mapping via OPENLAYER_LANGFLOW_MAPPING, and the priority order all line up with _get_config. Disable behavior via removing OPENLAYER_API_KEY is also consistent with the tracer’s readiness checks. Looks good.

src/backend/base/langflow/services/tracing/service.py (1)

62-66: Openlayer tracer wiring into the tracing lifecycle looks correct

_get_openlayer_tracer and _initialize_openlayer_tracer mirror the existing tracer helpers, are gated by self.deactivated, and registration under "openlayer" integrates cleanly with the existing tracers dict, ready checks, and LangChain callback aggregation. The addition to start_tracers is in the right place and won’t interfere with existing tracers.

Also applies to: 229-240, 269-269

[viniciusdsmello]

Hi @edwinjosechittilappilly, @ogabrielluiz and @Yukiyukiyeah. CodeRabbit suggested you as reviewers on this PR. Could any of you help us here?

Pls let me know if there's anything needed to improve this PR. I've already addressed CodeRabbit comments.

[viniciusdsmello]

@ogabrielluiz, can you merge this one?