Skip to content

feat(lfx): add adapter registries#11990

Merged
jordanrfrazier merged 23 commits into
mainfrom
lfx-subservice
Mar 5, 2026
Merged

feat(lfx): add adapter registries#11990
jordanrfrazier merged 23 commits into
mainfrom
lfx-subservice

Conversation

@HzaRashid
Copy link
Copy Markdown
Collaborator

@HzaRashid HzaRashid commented Mar 3, 2026

Summary

  • Introduce AdapterRegistry, a generic plugin registry that manages multiple swappable adapter
    implementations per adapter type (e.g. deployment adapters keyed by "local", "remote").
    Supports the same three discovery sources as ServiceManager — decorator registration, entry
    points, and TOML config files — with the same effective precedence (config > decorators > entry
    points).
  • Move the deployment service package from lfx/services/deployment/ to
    lfx/services/adapters/deployment/ and add AdapterType.DEPLOYMENT as the first adapter type.
    Expose a typed get_deployment_adapter() helper in lfx.services.deps for singleton resolution.
  • Extract shared TOML config discovery helpers (config_discovery.py) used by both
    ServiceManager and AdapterRegistry, removing duplicated config-loading logic from the
    service manager.

Motivation

The existing ServiceManager enforces one implementation per ServiceType. Some service
categories (like deployment) need multiple co-existing implementations selected by a runtime key.
Adapter registries fill this gap: they provide per-key class registration, lazy singleton
instantiation, and lifecycle management (teardown) without replacing the top-level service manager.

Key changes

  • lfx/services/adapters/registry.pyAdapterRegistry class, @register_adapter
    decorator, get_adapter_registry() singleton factory, and teardown_all_adapter_registries()
  • lfx/services/schema.pyAdapterType enum (DEPLOYMENT)
  • lfx/services/config_discovery.py — shared get_preferred_config_source,
    load_toml_config, get_nested_section
  • lfx/services/manager.py — refactored to use shared config helpers; calls adapter
    teardown during shutdown
  • lfx/services/deps.pyget_deployment_adapter() typed accessor
  • PLUGGABLE_SERVICES.md — documentation for adapter registries

Summary by CodeRabbit

  • New Features

    • Introduced an adapter registry system enabling service-scoped plugin discovery and management.
    • Added support for deployment service adapters discoverable from configuration files and entry points.
    • New public APIs: register_adapter, get_deployment_adapter, teardown_all_adapter_registries.
  • Documentation

    • Added Adapter Registries section to PLUGGABLE_SERVICES.md with API usage, registration methods, and integration patterns.
  • Tests

    • Added comprehensive unit tests for adapter registry discovery, lifecycle management, and concurrency scenarios.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Mar 3, 2026

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: c02715b4-2ecc-4d11-9640-24fa7ac46195

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Walkthrough

The changes introduce a service-scoped adapter registry system enabling dynamic discovery and management of swappable service implementations from decorators, entry points, and TOML configuration files, with thread-safe singleton lifecycle management and comprehensive public APIs for adapter registration and retrieval.

Changes

Cohort / File(s) Summary
Documentation & Type Definitions
src/lfx/PLUGGABLE_SERVICES.md, src/lfx/src/lfx/services/schema.py
Added Adapter Registries subsection documenting registration patterns, discovery flow, and lifecycle behavior. Introduced AdapterType enum with DEPLOYMENT member for adapter categorization.
Adapter Registry Infrastructure
src/lfx/src/lfx/services/adapters/registry.py, src/lfx/src/lfx/services/config_discovery.py
Implemented generic AdapterRegistry class supporting thread-safe singleton creation, discovery from entry points and TOML configs, and lifecycle management. Added config_discovery utilities for TOML loading and nested section resolution.
Service Integration & Exposure
src/lfx/src/lfx/services/__init__.py, src/lfx/src/lfx/services/deps.py, src/lfx/src/lfx/services/manager.py
Exposed new public APIs: register_adapter, get_deployment_adapter, teardown_all_adapter_registries. Integrated adapter registry discovery into dependency resolution. Updated service manager to invoke adapter teardown and use unified config loading utilities.
Deployment Adapter Migration
src/lfx/src/lfx/services/adapters/deployment/..., src/lfx/src/lfx/services/interfaces.py
Moved deployment service to adapters.deployment namespace; updated imports, docstrings, and exception references. Updated TYPE_CHECKING imports to reference new schema paths.
Test Infrastructure & Helpers
src/lfx/tests/unit/services/adapter_test_helpers.py, src/lfx/tests/unit/services/conftest.py
Added DeploymentAdapterStub test class and registry factory. Introduced clean_adapter_globals fixture for per-test adapter registry isolation.
Comprehensive Adapter Registry Tests
src/lfx/tests/unit/services/test_adapter_registry_*.py, src/lfx/tests/unit/services/deployment/test_deployment_*.py
Added unit tests covering registry concurrency, discovery from multiple sources, lifecycle/teardown behavior, conflict resolution, and decorator/entry-point/config precedence. Updated deployment tests to import from new adapters.deployment paths.

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant Decorator
    participant Registry as AdapterRegistry
    participant EntryPoints as Entry Points Loader
    participant Config as Config Loader
    participant Factory
    participant Adapter as Adapter Instance

    Client->>Registry: get_adapter_registry(adapter_type)
    activate Registry
    
    Note over Registry: Check global registry cache
    alt Registry exists
        Registry->>Client: return cached registry
    else Registry not cached
        Registry->>Registry: create new AdapterRegistry
        Registry->>Client: return new registry
    end
    deactivate Registry

    Client->>Decorator: `@register_adapter`(type, key)
    Decorator->>Registry: register_class(key, adapter_class, override=True)
    activate Registry
    Registry->>Registry: store adapter_class
    deactivate Registry

    Client->>Registry: discover(config_dir)
    activate Registry
    
    rect rgba(100, 150, 200, 0.5)
        Note over EntryPoints: Entry points discovery
        Registry->>EntryPoints: load from entry_point_group
        EntryPoints->>Registry: register_class(key, class, override=False)
    end

    rect rgba(150, 100, 200, 0.5)
        Note over Config: Config file discovery
        Registry->>Config: load lfx.toml or pyproject.toml
        Config->>Registry: register_class(key, class, override=True)
    end
    
    Registry->>Registry: mark discovered=True
    deactivate Registry

    Client->>Registry: get_instance(key, factory=factory_fn)
    activate Registry
    
    alt Instance already cached
        Registry->>Client: return cached instance
    else Not cached
        Registry->>Factory: factory(adapter_class)
        activate Factory
        Factory->>Adapter: create instance
        deactivate Factory
        Registry->>Registry: cache instance
        Registry->>Client: return instance
    end
    deactivate Registry

    Client->>Registry: teardown_instances()
    activate Registry
    Registry->>Adapter: await adapter.teardown()
    Registry->>Registry: clear instance cache
    deactivate Registry
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

🚥 Pre-merge checks | ✅ 5 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 43.30% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Excessive Mock Usage Warning ❓ Inconclusive The provided content appears to be shell script commands for reading test files, but no actual test file contents or verification results were provided to analyze. Please provide the actual output or contents of the test files being examined so verification can be performed.
✅ Passed checks (5 passed)
Check name Status Explanation
Test Coverage For New Implementations ✅ Passed The PR includes comprehensive test coverage with 30+ test functions across multiple test files (lifecycle, discovery, concurrency) with substantial assertions and mock setup for the new AdapterRegistry and config discovery functionality.
Test Quality And Coverage ✅ Passed PR includes comprehensive test coverage with 25+ test functions (353+ discovery, 91+ lifecycle, 42+ concurrency lines) validating all six public AdapterRegistry methods using proper pytest idioms, 44+ assertions, and thorough isolation.
Test File Naming And Structure ✅ Passed Backend and frontend tests follow standard naming conventions with appropriate organization and comprehensive coverage of edge cases and error conditions.
Title check ✅ Passed The title 'feat(lfx): add adapter registries' accurately describes the main change—introduction of a generic adapter registry system for service-scoped plugin resolution. It is concise, clear, and directly reflects the primary objective of the PR.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch lfx-subservice

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions github-actions Bot added the enhancement New feature or request label Mar 3, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 3, 2026

Frontend Unit Test Coverage Report

Coverage Summary

Lines Statements Branches Functions
Coverage: 23%
23.24% (8235/35429) 16.02% (4458/27826) 15.91% (1185/7445)

Unit Test Results

Tests Skipped Failures Errors Time
2631 0 💤 0 ❌ 0 🔥 44.879s ⏱️

@github-actions github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Mar 3, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 3, 2026

Codecov Report

❌ Patch coverage is 91.63763% with 24 lines in your changes missing coverage. Please review.
✅ Project coverage is 37.61%. Comparing base (6c4f1dd) to head (2cdbaa0).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
src/lfx/src/lfx/services/adapters/registry.py 92.63% 11 Missing and 1 partial ⚠️
src/lfx/src/lfx/services/config_discovery.py 89.65% 6 Missing ⚠️
src/lfx/src/lfx/services/manager.py 83.87% 5 Missing ⚠️
src/lfx/src/lfx/services/deps.py 95.00% 0 Missing and 1 partial ⚠️

❌ Your project status has failed because the head coverage (42.83%) is below the target coverage (60.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main   #11990      +/-   ##
==========================================
+ Coverage   37.44%   37.61%   +0.16%     
==========================================
  Files        1616     1619       +3     
  Lines       79060    79289     +229     
  Branches    11946    11971      +25     
==========================================
+ Hits        29607    29827     +220     
- Misses      47795    47803       +8     
- Partials     1658     1659       +1     
Flag Coverage Δ
backend 57.44% <ø> (+0.05%) ⬆️
frontend 20.82% <ø> (ø)
lfx 42.83% <91.63%> (+0.46%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...c/lfx/src/lfx/services/adapters/deployment/base.py 100.00% <ø> (ø)
...src/lfx/services/adapters/deployment/exceptions.py 100.00% <ø> (ø)
...lfx/src/lfx/services/adapters/deployment/schema.py 94.11% <100.00%> (ø)
...fx/src/lfx/services/adapters/deployment/service.py 100.00% <100.00%> (ø)
src/lfx/src/lfx/services/adapters/schema.py 100.00% <100.00%> (ø)
src/lfx/src/lfx/services/interfaces.py 100.00% <ø> (ø)
src/lfx/src/lfx/services/schema.py 100.00% <100.00%> (ø)
src/lfx/src/lfx/services/deps.py 66.33% <95.00%> (+6.58%) ⬆️
src/lfx/src/lfx/services/manager.py 79.35% <83.87%> (-2.44%) ⬇️
src/lfx/src/lfx/services/config_discovery.py 89.65% <89.65%> (ø)
... and 1 more

... and 7 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@HzaRashid HzaRashid changed the title feat(lfx): add subservice registry enabling multiple coexisting adapters feat(lfx): add sub-service registry for shared service-type plugins Mar 3, 2026
@github-actions github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Mar 3, 2026
@HzaRashid HzaRashid changed the title feat(lfx): add sub-service registry for shared service-type plugins feat(lfx): add sub-service registry for same service-type plugins Mar 3, 2026
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
src/lfx/tests/unit/services/test_subservice_registry.py (1)

136-142: Add a regression test for decorator override=False precedence.

Current tests cover SubServiceRegistry.register_sub_service_class(..., override=False) but not register_sub_service(..., override=False) when an entry point already registered the same key.

🧪 Suggested test addition
+def test_decorator_override_false_preserves_entrypoint(tmp_path, monkeypatch):
+    registry = _registry()
+    monkeypatch.setattr(
+        "importlib.metadata.entry_points",
+        lambda group: [DummyEntryPoint("local", Path)] if group == "lfx.deployment.adapters" else [],
+    )
+    subservice_mod.register_sub_service("deployment.adapters", "local", override=False)(PurePath)
+    registry.discover_sub_services(config_dir=tmp_path)
+    assert registry.get_sub_service_class("local") is Path

As per coding guidelines "Consider including edge cases and error conditions for comprehensive test coverage".

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/lfx/tests/unit/services/test_subservice_registry.py` around lines 136 -
142, Add a regression test that exercises
SubServiceRegistry.register_sub_service(..., override=False) when an entry point
(or prior register_sub_service_class call) already exists for the same key:
create a registry, register a class or entry for key "local" (e.g., via
register_sub_service_class("local", Path) or by simulating an entry point), then
call register_sub_service("local", some_factory_or_callable, override=False) and
assert the original registered class/factory remains (use
registry.get_sub_service_class("local") or registry.get_sub_service("local") as
appropriate); this mirrors the existing test for register_sub_service_class
override behavior but targets register_sub_service to prevent regressions.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/lfx/src/lfx/services/subservice.py`:
- Around line 23-24: The discovery replay currently forces override=True when
re-registering services, breaking calls that used register_sub_service(...,
override=False); update the replay logic to preserve the original override
semantics by reading and passing the stored override flag from
_decorator_subservice_registry when invoking register_sub_service (and similar
replay calls), rather than hardcoding True; touch all places noted
(_decorator_subservice_registry usage, the register_sub_service calls around the
blocks referencing _subservice_registries and SubServiceRegistry) so replayed
registrations honor the originally recorded override value.

---

Nitpick comments:
In `@src/lfx/tests/unit/services/test_subservice_registry.py`:
- Around line 136-142: Add a regression test that exercises
SubServiceRegistry.register_sub_service(..., override=False) when an entry point
(or prior register_sub_service_class call) already exists for the same key:
create a registry, register a class or entry for key "local" (e.g., via
register_sub_service_class("local", Path) or by simulating an entry point), then
call register_sub_service("local", some_factory_or_callable, override=False) and
assert the original registered class/factory remains (use
registry.get_sub_service_class("local") or registry.get_sub_service("local") as
appropriate); this mirrors the existing test for register_sub_service_class
override behavior but targets register_sub_service to prevent regressions.

ℹ️ Review info

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 91c016a and 51ec780.

📒 Files selected for processing (3)
  • src/lfx/PLUGGABLE_SERVICES.md
  • src/lfx/src/lfx/services/subservice.py
  • src/lfx/tests/unit/services/test_subservice_registry.py

Comment thread src/lfx/src/lfx/services/subservice.py Outdated
@github-actions github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Mar 3, 2026
@HzaRashid HzaRashid marked this pull request as draft March 3, 2026 04:03
@github-actions github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Mar 3, 2026
@github-actions github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Mar 3, 2026
@HzaRashid HzaRashid changed the title feat(lfx): add sub-service registry for same service-type plugins feat(lfx): add adapter registries for service-scoped plugin resolution Mar 3, 2026
@github-actions github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Mar 4, 2026
self._adapter_instances[key] = instance
return instance

async def teardown_instances(self) -> None:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not keep the lock open for the entire process? that could create a window for instances to be created while teardown is happening.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should probably add this here since we have very similar classes that could drift:

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from lfx.services.interfaces import DeploymentServiceProtocol

    # Static assertion: ensure ABC stays in sync with the Protocol.
    _: type[DeploymentServiceProtocol] = BaseDeploymentService

@ogabrielluiz
Copy link
Copy Markdown
Contributor

You should probably add a concurrent discovery test. The _discovered flag + lock in discover() isn't tested under contention. The concurrency test file only covers get_instance. Something like 8 threads all calling discover() and asserting the inner logic runs exactly once would be good to have.

HzaRashid and others added 21 commits March 5, 2026 14:38
- Use asyncio.iscoroutine for teardown (matches ServiceManager)
- Two-tier entry point error handling (warning vs debug)
- Add config load logging after adapter discovery
- Remove _get_nested_section wrapper, call shared helper directly
- Remove deployment/registry.py thin wrapper
- Export DeploymentServiceProtocol from services/__init__
- Extract test stubs into shared adapter_test_helpers module
- Revert incorrect ValueError->TypeError in schema validators
…ention

Align adapter registry with the service manager's @register_service
pattern: decorators now write directly to the AdapterRegistry singleton
instead of buffering in a module-level staging dict.

- register_adapter calls get_adapter_registry() + register_class()
  directly, removing _decorator_adapter_registry and _decorator_lock
- get_adapter_registry derives entry_point_group and
  config_section_path from AdapterType by convention, making them
  optional parameters
- Remove _discover_from_decorators() since decorators are already
  registered before discover() runs
- Fix discover_plugins docstring that incorrectly claimed decorators
  had highest priority (config files do)
- Simplify _reset_registries, deps.py, test helpers, and docs
…apter registry

  - Encapsulate AdapterRegistry internal state behind private attrs and
    read-only properties (adapter_type, entry_point_group, config_section_path,
    is_discovered, has_cached_instances)
  - Re-raise unexpected exceptions in register_adapter decorator instead of
    silently swallowing them
  - Promote entry-point discovery catch-all from debug to warning with traceback
  - Wrap register_class in RLock for thread safety
  - Add try/except with context logging around factory calls in get_instance
  - Improve teardown_instances: preserve keys for error messages, add exc_info
  - Evict stale cached instances when register_class changes the class for a key
  - Narrow load_toml_config exception to ValueError (parent of TOMLDecodeError)
  - Split load_object_from_import_path error handling: expected import failures
    vs unexpected errors with traceback
  - Guard redundant discover() calls in get_deployment_adapter via is_discovered
  - Add tests for teardown exception isolation, sync teardown, and no-teardown
    adapters
Comment thread src/lfx/tests/unit/services/test_adapter_registry_concurrency.py Outdated
Copy link
Copy Markdown
Contributor

@ogabrielluiz ogabrielluiz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Well-structured adapter registry with solid test coverage and clean design.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request lgtm This PR has been approved by a maintainer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants