-
Notifications
You must be signed in to change notification settings - Fork 149
Control Orchestrator - Unified Controller for multi-arm and full body controller #970
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
42 commits
Select commit
Hold shift + click to select a range
1d6b086
archive old driver to manipulators_old for redesign
mustafab0 e2ba169
spec.py defining minimal protocol for an arm driver
mustafab0 919691c
xarm driver driver added - driver owns control thread and robot state…
mustafab0 41d7b82
xarm SDK specific wrapper to interface with dimos RPC calls from the …
mustafab0 a95037c
removed type checking for old armdriver spec from the cartesian cont…
mustafab0 885cc4b
replicated piper driver to meet the new architecture
mustafab0 1472ecd
added mock backend
mustafab0 6e2a20d
updated all blueprints to add new arm module
mustafab0 c427fa5
Added readme explaining new driver architecture overview
mustafab0 f3b95f5
config now parsed in backend init instead of connect method
mustafab0 d6ea830
addded dual arm control blueprint using trajectory controller
mustafab0 f69337d
adding a control orchestrator for single control loop for multiple ar…
mustafab0 5265191
hardware interface protocol that wraps specific arm SDK to work with …
mustafab0 3500ba5
main orchestrator module and control loop that claims resources compu…
mustafab0 f33e2c7
added a trajectory task implementation that performs trajecotry control
mustafab0 b36f13b
added blueprints to launch orchestratory module with differnt arms fo…
mustafab0 cb9d87c
updated blueprints to add piper + xarm blueprint
mustafab0 300eca2
orchestrator client that can send tasks to the control orchestrator m…
mustafab0 b449732
added a readme
mustafab0 7b0e224
added pytest and e2e test
mustafab0 116d9a6
Update dimos/control/hardware_interface.py
mustafab0 0a8865f
CI code cleanup
mustafab0 17d8453
Fixed issues flagged by greptile
mustafab0 00de446
Renamed deprecated old manipuialtion test file and Mypy type fixes
mustafab0 72caf16
fix mypy test
mustafab0 6603cc5
mypy test fix added explicit type
mustafab0 a0400fa
Remove deprecated manipulators_old folder
mustafab0 142105b
fixed redef error in dual trajectory setter
mustafab0 1c442f7
Fixed bugs identified by greptile overview:
mustafab0 959e6da
addressed greptile suggestion:
mustafab0 67ca8eb
undo change to pyproject.toml
mustafab0 5fef952
Replaced _running bool with threading.Event (_stop_event) for thread …
mustafab0 81f4123
added type ignore for ipythin
mustafab0 449960a
removed check for has attribute in hardware interface
mustafab0 8f76a09
orchestrator.py: Use match statement for backend factory, restructure…
mustafab0 6143af6
tick_loop.py: Notify preemption when lower-priority task loses to exi…
mustafab0 60304d3
tick_loop.py: Notify preemption when lower-priority task loses to exi…
mustafab0 33e18a6
cleaned up legacy blueprints for manipulator drivers
mustafab0 230b8d7
enforce ManipulatorBackend Protocol on the backend.py
mustafab0 3216509
feat: add runtime protocol checks for manipulator backends
spomichter 9051c92
added runtime checking for controlTask protocol
spomichter 26d732b
Add TaskStatus dataclass, refactor get_trajectory_status and Explicit…
mustafab0 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,195 @@ | ||
| # Control Orchestrator | ||
|
|
||
| Centralized control system for multi-arm robots with per-joint arbitration. | ||
|
|
||
| ## Architecture | ||
|
|
||
| ``` | ||
| ┌─────────────────────────────────────────────────────────────┐ | ||
| │ ControlOrchestrator │ | ||
| │ │ | ||
| │ ┌──────────────────────────────────────────────────────┐ │ | ||
| │ │ TickLoop (100Hz) │ │ | ||
| │ │ │ │ | ||
| │ │ READ ──► COMPUTE ──► ARBITRATE ──► ROUTE ──► WRITE │ │ | ||
| │ └──────────────────────────────────────────────────────┘ │ | ||
| │ │ │ │ │ │ | ||
| │ ▼ ▼ ▼ ▼ │ | ||
| │ ┌─────────┐ ┌───────┐ ┌─────────┐ ┌──────────┐ │ | ||
| │ │Hardware │ │ Tasks │ │Priority │ │ Backends │ │ | ||
| │ │Interface│ │ │ │ Winners │ │ │ │ | ||
| │ └─────────┘ └───────┘ └─────────┘ └──────────┘ │ | ||
| └─────────────────────────────────────────────────────────────┘ | ||
| ``` | ||
|
|
||
| ## Quick Start | ||
|
|
||
| ```bash | ||
| # Terminal 1: Run orchestrator | ||
| dimos run orchestrator-mock # Single 7-DOF mock arm | ||
| dimos run orchestrator-dual-mock # Dual arms (7+6 DOF) | ||
| dimos run orchestrator-piper-xarm # Real hardware | ||
|
|
||
| # Terminal 2: Control via CLI | ||
| python -m dimos.manipulation.control.orchestrator_client | ||
| ``` | ||
|
|
||
| ## Core Concepts | ||
|
|
||
| ### Tick Loop | ||
| Single deterministic loop at 100Hz: | ||
| 1. **Read** - Get joint positions from all hardware | ||
| 2. **Compute** - Each task calculates desired output | ||
| 3. **Arbitrate** - Per-joint, highest priority wins | ||
| 4. **Route** - Group commands by hardware | ||
| 5. **Write** - Send commands to backends | ||
|
|
||
| ### Tasks (Controllers) | ||
| Tasks are passive controllers called by the orchestrator: | ||
|
|
||
| ```python | ||
| class MyController: | ||
| def claim(self) -> ResourceClaim: | ||
| return ResourceClaim(joints={"joint1", "joint2"}, priority=10) | ||
|
|
||
| def compute(self, state: OrchestratorState) -> JointCommandOutput: | ||
| # Your control law here (PID, impedance, etc.) | ||
| return JointCommandOutput( | ||
| joint_names=["joint1", "joint2"], | ||
| positions=[0.5, 0.3], | ||
| mode=ControlMode.POSITION, | ||
| ) | ||
| ``` | ||
|
|
||
| ### Priority & Arbitration | ||
| Higher priority always wins. Arbitration happens every tick: | ||
|
|
||
| ``` | ||
| traj_arm (priority=10) wants joint1 = 0.5 | ||
| safety (priority=100) wants joint1 = 0.0 | ||
| ↓ | ||
| safety wins, traj_arm preempted | ||
| ``` | ||
|
|
||
| ### Preemption | ||
| When a task loses a joint to higher priority, it gets notified: | ||
|
|
||
| ```python | ||
| def on_preempted(self, by_task: str, joints: frozenset[str]) -> None: | ||
| self._state = TrajectoryState.PREEMPTED | ||
| ``` | ||
|
|
||
| ## Files | ||
|
|
||
| ``` | ||
| dimos/control/ | ||
| ├── orchestrator.py # Module + RPC interface | ||
| ├── tick_loop.py # 100Hz control loop | ||
| ├── task.py # ControlTask protocol + types | ||
| ├── hardware_interface.py # Backend wrapper | ||
| ├── blueprints.py # Pre-configured setups | ||
| └── tasks/ | ||
| └── trajectory_task.py # Joint trajectory controller | ||
| ``` | ||
|
|
||
| ## Configuration | ||
|
|
||
| ```python | ||
| from dimos.control import control_orchestrator, HardwareConfig, TaskConfig | ||
|
|
||
| my_robot = control_orchestrator( | ||
| tick_rate=100.0, | ||
| hardware=[ | ||
| HardwareConfig(id="left", type="xarm", dof=7, joint_prefix="left", ip="192.168.1.100"), | ||
| HardwareConfig(id="right", type="piper", dof=6, joint_prefix="right", can_port="can0"), | ||
| ], | ||
| tasks=[ | ||
| TaskConfig(name="traj_left", type="trajectory", joint_names=[...], priority=10), | ||
| TaskConfig(name="traj_right", type="trajectory", joint_names=[...], priority=10), | ||
| TaskConfig(name="safety", type="trajectory", joint_names=[...], priority=100), | ||
| ], | ||
| ) | ||
| ``` | ||
|
|
||
| ## RPC Methods | ||
|
|
||
| | Method | Description | | ||
| |--------|-------------| | ||
| | `list_hardware()` | List hardware IDs | | ||
| | `list_joints()` | List all joint names | | ||
| | `list_tasks()` | List task names | | ||
| | `get_joint_positions()` | Get current positions | | ||
| | `execute_trajectory(task, traj)` | Execute trajectory | | ||
| | `get_trajectory_status(task)` | Get task status | | ||
| | `cancel_trajectory(task)` | Cancel active trajectory | | ||
|
|
||
| ## Control Modes | ||
|
|
||
| Tasks output commands in one of three modes: | ||
|
|
||
| | Mode | Output | Use Case | | ||
| |------|--------|----------| | ||
| | POSITION | `q` | Trajectory following | | ||
| | VELOCITY | `q_dot` | Joystick teleoperation | | ||
| | TORQUE | `tau` | Force control, impedance | | ||
|
|
||
| ## Writing a Custom Task | ||
|
|
||
| ```python | ||
| from dimos.control.task import ControlTask, ResourceClaim, JointCommandOutput, ControlMode | ||
|
|
||
| class PIDController: | ||
| def __init__(self, joints: list[str], priority: int = 10): | ||
| self._name = "pid_controller" | ||
| self._claim = ResourceClaim(joints=frozenset(joints), priority=priority) | ||
| self._joints = joints | ||
| self.Kp, self.Ki, self.Kd = 10.0, 0.1, 1.0 | ||
| self._integral = [0.0] * len(joints) | ||
| self._last_error = [0.0] * len(joints) | ||
| self.target = [0.0] * len(joints) | ||
|
|
||
| @property | ||
| def name(self) -> str: | ||
| return self._name | ||
|
|
||
| def claim(self) -> ResourceClaim: | ||
| return self._claim | ||
|
|
||
| def is_active(self) -> bool: | ||
| return True | ||
|
|
||
| def compute(self, state) -> JointCommandOutput: | ||
| positions = [state.joints.joint_positions[j] for j in self._joints] | ||
| error = [t - p for t, p in zip(self.target, positions)] | ||
|
|
||
| # PID | ||
| self._integral = [i + e * state.dt for i, e in zip(self._integral, error)] | ||
| derivative = [(e - le) / state.dt for e, le in zip(error, self._last_error)] | ||
| output = [self.Kp*e + self.Ki*i + self.Kd*d | ||
| for e, i, d in zip(error, self._integral, derivative)] | ||
| self._last_error = error | ||
|
|
||
| return JointCommandOutput( | ||
| joint_names=self._joints, | ||
| positions=output, | ||
| mode=ControlMode.POSITION, | ||
| ) | ||
|
|
||
| def on_preempted(self, by_task: str, joints: frozenset[str]) -> None: | ||
| pass # Handle preemption | ||
| ``` | ||
|
|
||
| ## Joint State Output | ||
|
|
||
| The orchestrator publishes one aggregated `JointState` message containing all joints: | ||
|
|
||
| ```python | ||
| JointState( | ||
| name=["left_joint1", ..., "right_joint1", ...], # All joints | ||
| position=[...], | ||
| velocity=[...], | ||
| effort=[...], | ||
| ) | ||
| ``` | ||
|
|
||
| Subscribe via: `/orchestrator/joint_state` | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,92 @@ | ||
| # Copyright 2025-2026 Dimensional Inc. | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| """ControlOrchestrator - Centralized control for multi-arm coordination. | ||
|
|
||
| This module provides a centralized control orchestrator that replaces | ||
| per-driver/per-controller loops with a single deterministic tick-based system. | ||
|
|
||
| Features: | ||
| - Single tick loop (read → compute → arbitrate → route → write) | ||
| - Per-joint arbitration (highest priority wins) | ||
| - Mode conflict detection | ||
| - Partial command support (hold last value) | ||
| - Aggregated preemption notifications | ||
|
|
||
| Example: | ||
| >>> from dimos.control import ControlOrchestrator | ||
| >>> from dimos.control.tasks import JointTrajectoryTask, JointTrajectoryTaskConfig | ||
| >>> from dimos.hardware.manipulators.xarm import XArmBackend | ||
| >>> | ||
| >>> # Create orchestrator | ||
| >>> orch = ControlOrchestrator(tick_rate=100.0) | ||
| >>> | ||
| >>> # Add hardware | ||
| >>> backend = XArmBackend(ip="192.168.1.185", dof=7) | ||
| >>> backend.connect() | ||
| >>> orch.add_hardware("left_arm", backend, joint_prefix="left") | ||
| >>> | ||
| >>> # Add task | ||
| >>> joints = [f"left_joint{i+1}" for i in range(7)] | ||
| >>> task = JointTrajectoryTask( | ||
| ... "traj_left", | ||
| ... JointTrajectoryTaskConfig(joint_names=joints, priority=10), | ||
| ... ) | ||
| >>> orch.add_task(task) | ||
| >>> | ||
| >>> # Start | ||
| >>> orch.start() | ||
| """ | ||
|
|
||
| from dimos.control.hardware_interface import ( | ||
| BackendHardwareInterface, | ||
| HardwareInterface, | ||
| ) | ||
| from dimos.control.orchestrator import ( | ||
| ControlOrchestrator, | ||
| ControlOrchestratorConfig, | ||
| HardwareConfig, | ||
| TaskConfig, | ||
| control_orchestrator, | ||
| ) | ||
| from dimos.control.task import ( | ||
| ControlMode, | ||
| ControlTask, | ||
| JointCommandOutput, | ||
| JointStateSnapshot, | ||
| OrchestratorState, | ||
| ResourceClaim, | ||
| ) | ||
| from dimos.control.tick_loop import TickLoop | ||
|
|
||
| __all__ = [ | ||
| # Hardware interface | ||
| "BackendHardwareInterface", | ||
| "ControlMode", | ||
| # Orchestrator | ||
| "ControlOrchestrator", | ||
| "ControlOrchestratorConfig", | ||
| # Task protocol and types | ||
| "ControlTask", | ||
| "HardwareConfig", | ||
| "HardwareInterface", | ||
| "JointCommandOutput", | ||
| "JointStateSnapshot", | ||
| "OrchestratorState", | ||
| "ResourceClaim", | ||
| "TaskConfig", | ||
| # Tick loop | ||
| "TickLoop", | ||
| "control_orchestrator", | ||
| ] |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Need to use @leshy docs stuff such that people can actually run code in the docs. I forget what its called but basicaly makes docs like jupyter notebook and checks that all code in docs is actually executable and passes tests. So docs code will never break with new PRs
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
#1024