Skip to content

Config adjustments#1369

Merged
paul-nechifor merged 19 commits intodevfrom
config-tweak
Mar 3, 2026
Merged

Config adjustments#1369
paul-nechifor merged 19 commits intodevfrom
config-tweak

Conversation

@Dreamsorcerer
Copy link
Collaborator

@Dreamsorcerer Dreamsorcerer commented Feb 25, 2026

Problem

Working towards #1082, this helps standardise setting in modules which can then be expanded upon to enable further things such as CLI arguments to be injected. They are also now type checked at runtime via Pydantic.

Solution

Primary change to developers is that the __init__() method now uses a fixed signature and all configurable options appear in the config. That config is now a pydantic model, so it gets verified when the module is instantiated.

A lot of typing fixes were also made by changing incorrect references to Module to ModuleBase and similar adjustments.

Breaking Changes

Subclasses of Module must now match the __init__() signature of the Module class.

@Dreamsorcerer Dreamsorcerer changed the base branch from main to dev February 25, 2026 17:14
setattr(cls, name, None)

def __init__(self, *args, **kwargs) -> None: # type: ignore[no-untyped-def]
def __init__(self, global_config: GlobalConfig = global_config, **kwargs: Any):
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As an alternative to asking subclasses (that need a custom __init__()) to include global_config here, I could move it into the ModuleConfig itself? Then subclasses will just pass kwargs alone.

@Dreamsorcerer Dreamsorcerer changed the title Config tweak Config adjustments Feb 26, 2026
@Dreamsorcerer Dreamsorcerer marked this pull request as ready for review February 26, 2026 18:51
@greptile-apps
Copy link
Contributor

greptile-apps bot commented Feb 26, 2026

Greptile Summary

This PR standardizes module configuration by migrating from dataclasses to Pydantic models and enforcing a fixed __init__() signature across all Module subclasses.

Key Changes:

  • All ModuleConfig classes now inherit from BaseConfig (Pydantic BaseModel) instead of using @dataclass
  • Module __init__() signature standardized to __init__(self, global_config: GlobalConfig = global_config, **kwargs: Any)
  • Configuration parameters moved from constructor arguments to config class fields
  • Runtime type checking now provided via Pydantic validation
  • Type annotations updated throughout (Module → ModuleBase, added generic TypeVars)
  • Infrastructure updated (Worker, ModuleCoordinator, Blueprint) to pass configs correctly

Issues Found:

  • TemporalMemory.__init__() doesn't follow the new pattern - still uses custom parameters instead of the standardized signature
  • Detection2DModule.camera_info changed from optional to required (documented breaking change, tests updated)

Impact:
Most modules properly migrated to the new pattern. The standardization will enable CLI argument injection and better configuration management going forward.

Confidence Score: 3/5

  • This PR is mostly safe but has one module that doesn't follow the new pattern
  • The refactoring is well-executed across 59 of 60 files with consistent pattern application and proper test updates. However, TemporalMemory module wasn't fully migrated and still uses the old __init__ signature, which violates the documented breaking change requirement. This will cause runtime errors when the module is instantiated through the standard deployment pipeline.
  • Pay close attention to dimos/perception/experimental/temporal_memory/temporal_memory.py - the __init__ method needs to be updated to match the new standardized signature before merging

Important Files Changed

Filename Overview
dimos/core/module.py Core module refactored to use fixed __init__ signature with Pydantic config, TypeVar updates for Python 3.13 compatibility
dimos/core/native_module.py Migrated from dataclass to Pydantic BaseModel, updated compute_cli_args() to use model_fields
dimos/protocol/service/spec.py Added BaseConfig as Pydantic BaseModel with arbitrary_types_allowed, updated Configurable class
dimos/core/worker.py Updated Worker to pass global_config and kwargs instead of args and kwargs, type annotations updated
dimos/core/module_coordinator.py Updated deploy methods to use new ModuleSpec tuple format, removed signature inspection logic
dimos/core/blueprints.py Removed args from _BlueprintAtom, removed signature inspection for cfg parameter
dimos/perception/experimental/temporal_memory/temporal_memory.py Type annotations updated but __init__ signature not updated to match new pattern - potential bug
dimos/perception/detection/module2D.py Migrated to Pydantic, camera_info now required (was optional), uses Pydantic validation for filter field
dimos/robot/unitree/b1/connection.py Properly migrated with new B1ConnectionConfig, __init__ updated correctly
dimos/models/base.py Converted LocalModelConfig and HuggingFaceModelConfig from dataclass to Pydantic BaseConfig
dimos/protocol/service/lcmservice.py Migrated LCMConfig to Pydantic, removed __post_init__, added TypeVar for generic config
dimos/core/worker_manager.py Updated deploy methods to pass global_config and kwargs instead of args tuple

Class Diagram

%%{init: {'theme': 'neutral'}}%%
classDiagram
    class BaseConfig {
        <<Pydantic BaseModel>>
        +model_config: arbitrary_types_allowed
    }
    
    class ModuleConfig {
        +rpc_transport: RPCSpec
        +tf_transport: TFSpec
        +frame_id_prefix: str | None
        +frame_id: str | None
    }
    
    class NativeModuleConfig {
        +executable: str
        +build_command: str | None
        +extra_args: list
        +extra_env: dict
        +compute_cli_args() list
    }
    
    class Configurable~ConfigT~ {
        <<Generic>>
        +default_config: type[ConfigT]
        +config: ConfigT
        +__init__(**kwargs)
    }
    
    class ModuleBase~ModuleConfigT~ {
        +default_config: ModuleConfig
        -_rpc: RPCSpec | None
        -_tf: TFSpec | None
        +__init__(config_args: dict, global_config: GlobalConfig)
    }
    
    class Module~ModuleConfigT~ {
        +__init__(global_config: GlobalConfig, **kwargs)
        +set_ref(ref)
        +start()
        +stop()
    }
    
    class NativeModule~NativeConfig~ {
        +default_config: NativeModuleConfig
        -_process: Popen | None
        +__init__(global_config: GlobalConfig, **kwargs)
    }
    
    BaseConfig <|-- ModuleConfig : inherits
    ModuleConfig <|-- NativeModuleConfig : inherits
    Configurable <|-- ModuleBase : inherits
    ModuleBase <|-- Module : inherits
    Module <|-- NativeModule : inherits
    
    Configurable --> BaseConfig : uses ConfigT
    ModuleBase --> ModuleConfig : uses ModuleConfigT
    
    note for ModuleBase "Fixed __init__ signature:\nconfig_args + global_config"
    note for Module "Standardized signature:\nglobal_config + **kwargs"
    note for BaseConfig "Runtime validation\nvia Pydantic"
Loading

Last reviewed commit: 28a43f7

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

60 files reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

paul-nechifor
paul-nechifor previously approved these changes Feb 27, 2026
@paul-nechifor
Copy link
Contributor

paul-nechifor commented Feb 27, 2026

The tests are still failing.

Note that you have to cherry-pick this to fix protobuf: https://github.com/dimensionalOS/dimos/pull/1372/changes (No longer needed, you just need to rebased on dev now.)

Sam Bull added 2 commits March 2, 2026 19:06
# TODO: All of these should be fixed, but it's easier commit autofixes first
"A001", "A002", "B008", "B017", "B019", "B024", "B026", "B904", "C901", "E402", "E501", "E721", "E722", "E741", "F811", "F821", "F821", "F821", "N801", "N802", "N803", "N806", "N817", "N999", "RUF003", "RUF009", "RUF012", "RUF034", "RUF043", "RUF059", "UP007",
# This breaks runtime type checking (both for us, and users introspecting our APIs)
"TC001", "TC002", "TC003"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fixed this recently. The linter needs to know which classes need to have their annotations at run time. Are there more classes besides Module ones now?

[tool.ruff.lint.flake8-type-checking]
runtime-evaluated-base-classes = ["dimos.core.module.Module"]

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess it's mainly ModuleConfig now. But, that may reference other classes which are not subclassed from ModuleConfig, creating a lot of extra false positives. I still get test failures after adding ModuleConfig and letting ruff fix things.

This rule is one of my pet peeves, as it achieves nothing (all these modules are getting imported at runtime somewhere, so there's going to be no noticeable improvement to import time), and thus only causing problems and making code slightly less readable. As noted in the comment, there's a small chance users may want to introspect our APIs at runtime and this would break any attempts to do that.

class ModuleConfig(BaseConfig):
rpc_transport: type[RPCSpec] = LCMRPC
tf_transport: type[TFSpec] = LCMTF
tf_transport: type[TFSpec] = LCMTF # type: ignore[type-arg]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure # type: ignore[type-arg] is needed? It wasn't before.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, pydantic won't let me resolve it:

Subscripting `type[]` with an already prametrized type is not supported.

It's because I made it generic, so mypy wants it subscripted.

paul-nechifor
paul-nechifor previously approved these changes Mar 3, 2026
@leshy
Copy link
Contributor

leshy commented Mar 3, 2026

hey for the future for easier reviews, usually we ask for oneliners to test, but this is an API change so could you show us the new API with example code blocks, like what does this change mean for us building modules.. either this or docs update link we can read

@leshy
Copy link
Contributor

leshy commented Mar 3, 2026

Also do we need to accept global_config as arg even if not using global config in our module? why not pass kwargs directly?

@Dreamsorcerer
Copy link
Collaborator Author

Also do we need to accept global_config as arg even if not using global config in our module? why not pass kwargs directly?

I think that goes back to my open question above:
#1369 (comment)

If we move it to the ModuleConfig, then subclasses don't need to worry about defining the parameter, and it's then always there if needed.

Most modules won't need to define a custom __init__ anyway, so shouldn't affect too many modules. It's basically only ones that need to initialise mutable attributes on creation.

Sam Bull added 2 commits March 3, 2026 19:03
@paul-nechifor paul-nechifor merged commit 790397c into dev Mar 3, 2026
12 checks passed
paul-nechifor added a commit that referenced this pull request Mar 4, 2026
paul-nechifor added a commit that referenced this pull request Mar 4, 2026
RadientBrain pushed a commit to RadientBrain/dimos that referenced this pull request Mar 4, 2026
* Adjust config definitions for future enhancements

* Fixes

* Update module.py

* Update module.py

* Update blueprints.py

* Fixes

* Fixes

* Fixes

* Apply suggestion from @Dreamsorcerer

* Update person_follow.py

* Update person_follow.py

* Fixes

* Fixes

* Fixes

* Fixes

---------

Co-authored-by: Sam Bull <Sam.B@snowfalltravel.com>
RadientBrain pushed a commit to RadientBrain/dimos that referenced this pull request Mar 4, 2026
RadientBrain pushed a commit to RadientBrain/dimos that referenced this pull request Mar 4, 2026
* Adjust config definitions for future enhancements

* Fixes

* Update module.py

* Update module.py

* Update blueprints.py

* Fixes

* Fixes

* Fixes

* Apply suggestion from @Dreamsorcerer

* Update person_follow.py

* Update person_follow.py

* Fixes

* Fixes

* Fixes

* Fixes

---------

Co-authored-by: Sam Bull <Sam.B@snowfalltravel.com>
RadientBrain pushed a commit to RadientBrain/dimos that referenced this pull request Mar 4, 2026
Dreamsorcerer pushed a commit that referenced this pull request Mar 4, 2026
leshy pushed a commit that referenced this pull request Mar 4, 2026
* Reapply "Config adjustments (#1369)" (#1409)

This reverts commit ae781c1.

* Fix

---------

Co-authored-by: Sam Bull <Sam.B@snowfalltravel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants