Feat/Quest VR teleoperation stack by ruthwikdasyam · Pull Request #1215 · dimensionalOS/dimos

ruthwikdasyam · 2026-02-07T23:40:39Z

Feature

Adding teleoperation support to DimOS stack. A way to control any robots (Arms, Quadrupeds, Humanoids) via teleop.
This PR sets foundation for how a device (VR, Xbox controllers, Exoskeleton, Gloves) must be laid out to expose its controls out - to be connected with any robot on dimos.
This PR has explicitly Implementation for the Quest3 VR headset.

Issue - Linear

closes DIM-420
closes DIM-394

closes #1185
closes #1134

Approach/Solution

Web stuff

WebXR captures controller poses + buttons data and sends over Websocket to Deno server
Deno HTTPS server bridge forwards same raw bytes over UDP multicast via LCM
python QuestTeleopModule subscribes to those LCM channels and decodes
Further, we can send LCM messages back through UDP for any Visualizations/Feedback to display in the VR

Architecture

QuestTeleopModule receives these Pose/Joy LCM streams. once engaged, computes deltas and publishes msg commands.
QuestControllerState/'QuestButtons' wraps over raw Joy messages for clean controller data access.
subclasses override hooks (_handle_engage, _should_publish, _get_output_pose, _publish_msg) to customize what commands get published to the robot.
Example implementations ArmTeleopModule (toggle-engage arm control), TwistTeleopModule (velocity commands), VisualizingTeleopModule (Rerun debug overlay) are included.

Refer for Detailed Spec

Teleop restructuring #1113 (Latest Spec at the end)

Breaking Changes

None

How to Test

If you have Quest3

Start the Deno bridge: ./dimos/teleop/quest/web/teleop_server.ts
In another terminal: dimos run arm-teleop-visualizing
On Quest 3, open https://<host-ip>:8443 and click connect - opens passthrough
Press X to engage

move controllers and press buttons and toggle
verify poses and data update in real-time in Rerun

Press X again to disengage - verify poses stop updating
To End: click menu button, closes passthrough view - click disconnect in VR browser

Files

assets/teleop_certs - for generated self-signed TLS certs
dimos/teleop/ - full subsystem (protocol, quest module, types, extensions, transforms, visualization, blueprints)
dimos/msgs/std_msgs/UInt32.py - base type for QuestButtons
dimos/teleop/quest/web/ - Deno bridge + WebXR client

greptile-apps · 2026-02-08T05:50:07Z

Greptile Overview

Greptile Summary

This PR adds a complete Meta Quest 3 teleoperation stack spanning:

WebXR client (dimos/teleop/quest/web/static/index.html) streaming PoseStamped and Joy over WebSocket at ~80Hz.
Deno bridge (dimos/teleop/quest/web/teleop_server.ts) serving the WebXR UI over HTTPS and forwarding raw LCM packets between browser and UDP LCM.
Python teleop module (dimos/teleop/quest/quest_teleop_module.py) that subscribes to VR pose/joy topics, transforms WebXR poses into the robot frame (teleop_transforms.py), computes controller deltas (via new Pose.__sub__), and publishes Pose/Twist outputs plus packed button state (QuestButtons).

The change fits into the codebase by registering new module entrypoints and blueprints (dimos/robot/all_blueprints.py, dimos/teleop/blueprints.py) so the teleop pipeline can be instantiated via the existing blueprint/module system.

Main issues to address before merge:

The Deno server references lcm in the request handler before it is initialized/started, causing runtime failures on first websocket traffic.
The Python Joy parsing path can raise unhandled ValueError (format mismatch), which can break the subscription callback.
The module’s conditional subscription logic (if stream.transport) can result in a “starts but never subscribes” behavior depending on how transports are attached.
QuestButtons bitmask mutation depends on data initialization semantics from the UInt32/LCM base class; ensure data is initialized before bit-ops are used.

Confidence Score: 2/5

This PR has merge-blocking runtime issues in the Deno bridge and robustness gaps in the Python input handling.
Score is reduced due to a definite runtime ordering bug in teleop_server.ts (server starts before LCM init), plus unhandled Joy format errors and wiring/subscription behavior that can make the module silently non-functional depending on transport setup.
dimos/teleop/quest/web/teleop_server.ts, dimos/teleop/quest/quest_teleop_module.py, dimos/teleop/quest/quest_types.py

Important Files Changed

Filename	Overview
.gitignore	Ignores generated teleop TLS cert directory under dimos/assets/teleop_certs/.
dimos/msgs/geometry_msgs/Pose.py	Adds Pose.sub to compute delta pose (position subtraction + quaternion delta).
dimos/msgs/std_msgs/UInt32.py	Adds ROS-compatible UInt32 wrapper; potential base-init concerns for subclasses relying on data being initialized.
dimos/teleop/blueprints.py	Adds arm teleop blueprints wiring Quest teleop outputs to LCM topics.
dimos/teleop/quest/quest_teleop_module.py	Implements core QuestTeleopModule with threads + RLock control loop; issues: uncaught ValueError in _on_joy, RPC/lock layering, and conditional subscription on transport.
dimos/teleop/quest/quest_types.py	Adds QuestControllerState parsing and QuestButtons bitmask; potential initialization/bitmask setattr pitfalls.
dimos/teleop/quest/web/teleop_server.ts	Adds Deno HTTPS+WebSocket to LCM bridge; critical bug: server starts before LCM is initialized, causing runtime failures.
dimos/teleop/utils/teleop_transforms.py	Adds WebXR-to-robot transform utility using matrices + controller-specific Z rotation.

Sequence Diagram

sequenceDiagram
    participant Quest as Quest Browser (WebXR)
    participant Deno as Deno teleop_server.ts
    participant LCM as LCM bus (UDP)
    participant Py as QuestTeleopModule (Python)

    Quest->>Deno: wss:// connect
    loop ~80Hz
        Quest->>Deno: WS binary (LCM packet)<br/>/vr_left_pose, /vr_right_pose
        Quest->>Deno: WS binary (LCM packet)<br/>/vr_left_joy, /vr_right_joy
        Deno->>LCM: publishPacket(packet)
        LCM-->>Py: PoseStamped / Joy
        Py->>Py: _on_pose/_on_joy
    end

    loop 50Hz control loop
        Py->>Py: _handle_engage
        Py->>Py: _get_output_pose (delta)
        Py->>LCM: publish PoseStamped (/teleop/*)
        Py->>LCM: publish QuestButtons (/teleop/buttons)
    end

    LCM-->>Deno: subscribePacket(raw)
    Deno-->>Quest: WS binary (LCM packet)

greptile-apps

_{8 files reviewed, 5 comments}

_{Edit Code Review Agent Settings | Greptile}

greptile-apps · 2026-02-08T05:50:28Z

Additional Comments (5)

dimos/teleop/quest/web/teleop_server.ts
Serve starts before LCM init

Deno.serve(...) is invoked before const lcm = new LCM(); await lcm.start(); (later in the file). The HTTP handler’s websocket path uses await lcm.publishPacket(packet), but lcm hasn’t been initialized yet at server start, so the first websocket upgrade/message will throw (and likely crash the request handler). Initialize/start lcm before calling Deno.serve, or move the server startup below the LCM initialization.

dimos/teleop/quest/quest_teleop_module.py
Deadlock via re-entrant lock

_control_loop() holds self._lock for the full iteration, and _handle_engage() calls self.engage()/self.disengage() which are @rpc methods that also acquire self._lock (re-entrant). With threading.RLock this won’t deadlock, but it will run the engage logic twice through two lock layers and can block other threads longer than intended. More importantly, in subclasses overriding _handle_engage() (e.g., ArmTeleopModule), you’re calling engage()/disengage() from inside the lock-held loop too. Consider adding non-RPC internal helpers (e.g., _set_engaged(hand, bool)) that assume the lock is held, and have RPC methods delegate to them, so the control loop doesn’t go through RPC wrappers.

dimos/teleop/quest/quest_teleop_module.py
Unhandled ValueError kills callback

QuestControllerState.from_joy() raises ValueError when Joy doesn’t have the expected axis/button lengths, but _on_joy() doesn’t catch it. If the WebXR client sends fewer buttons/axes (common across browsers/firmware), the subscription callback will raise and can terminate/disable that stream. Catch ValueError here (log once/throttle) and ignore the malformed message so the module stays alive.

dimos/teleop/quest/quest_teleop_module.py
Inputs silently not subscribed

The if stream and stream.transport guard means the module won’t subscribe unless the In[...] already has a transport attached at start(). In typical blueprint wiring, the transport may be attached after instantiation but before start, but if it’s attached later (or is lazily created), this module will never subscribe and will never receive data. Subscribing unconditionally (and letting .subscribe raise if miswired) or rechecking/attaching on transport changes would prevent a “starts but does nothing” failure mode.

dimos/teleop/quest/quest_types.py
Bitmask attr set may break

QuestButtons.__setattr__ mutates self.data via self.data |= ... / &= .... If the underlying LCMUInt32 base class implements data as a property/slot without a pre-existing data attribute at construction, this can raise during from_controllers() when buttons = cls() and then buttons.left_trigger = ... runs. Ensure UInt32.__init__ runs (and sets data) for subclasses, or explicitly initialize data in QuestButtons.__init__ before using bit ops.

ruthwikdasyam · 2026-02-08T06:35:08Z

All Greptile feedback addressed in latest commits

paul-nechifor · 2026-02-08T06:55:51Z

.gitignore

 *mobileclip*
 /results
+
+dimos/assets/teleop_certs/


dimos/ is only for code. Should this have been assets/teleop_certs/?

Good catch, fixed this
server path + .gitignore are updated to /assets/teleop_certs/

paul-nechifor · 2026-02-08T07:05:14Z

dimos/teleop/quest/web/README.md

+## Running
+
+```bash
+deno run --allow-net --allow-read --allow-run --allow-write --unstable-net dimos/teleop/quest/web/teleop_server.ts


Since you have the #!/usr/bin/env -S deno run --allow-net --allow-read --allow-run --allow-write --unstable-net she-bang there, couldn't this be just:

./dimos/teleop/quest/web/teleop_server.ts

Yep, updated the README and added the execute bit.
Git tracks the file mode, so it'll be executable for all after this commit.

leshy · 2026-02-08T22:12:45Z

dimos/teleop/quest/web/teleop_server.ts

+console.log(`Server: https://localhost:${PORT}`);
+
+// Forward all raw packets to browser (we are decoding LCM directly in the browser)
+lcm.subscribePacket((packet) => {


oof you are sending ALL of the dimos pubsub packets into quest, this is A LOT of data that's not being processed at all, just loads the CPU and I assume will crash on applications that involve cameras or sensors

leshy · 2026-02-08T22:13:41Z

dimos/teleop/quest/web/static/index.html

+                ws.onclose = () => {
+                    setStatus('WebSocket closed');
+                };
+                ws.onmessage = () => {};


as you are forwarding every pubsub message, this function is actually called at hundreds or thousands of times a second

leshy · 2026-02-08T22:14:20Z

dimos/teleop/quest/web/static/index.html

+                        buttons.push(gamepad.buttons[i]?.pressed ? 1 : 0);
+                    }
+
+                    const joyMsg = new sensor_msgs.Joy({


nice, yes, that looks like a correct msg

leshy

there is this issue with forwarding all LCM packets to quest otherwise looks good.

Generally for PRs pls include an example how to run this so people can test easily, I'm missing for example an XAarm teleop blueprint or something to make it clear how this is to be used

leshy · 2026-02-08T22:18:14Z

dimos/teleop/quest/web/README.md

+Deno server that bridges WebSocket and LCM:
+- Serves WebXR client over HTTPS (required for Quest)
+- Forwards controller data from browser to LCM
+- Forwards LCM packets to browser


it shouldn't forward any LCM packets to browser right?

Okay yes. Was trying out sending images to display them in the headset. This PR doesn't have it - not forwarding any messages to the browser. Let me remove the lines related to it. Thanks for pointing it out!

leshy · 2026-02-08T22:19:20Z

dimos/teleop/quest/web/static/index.html

+            document.getElementById('status').textContent = `Error: ${msg}`;
+        };
+
+        import { encodePacket, geometry_msgs, std_msgs, sensor_msgs } from "https://esm.sh/jsr/@dimos/msgs@0.1.4";


all good, just so we are aware, dependancy on the external package sstore makes the system not work without an internet connection on quest

yeah, its loading the msgs package from an external CDN at runtime. can mention this somewhere in README for now.

leshy · 2026-02-08T22:20:30Z

dimos/teleop/quest/web/teleop_server.ts

@@ -0,0 +1,95 @@
+#!/usr/bin/env -S deno run --allow-net --allow-read --allow-run --allow-write --unstable-net


for the next pass, generally I'd want to auto-start this somehow as a part of the blueprint, so I don't need an extra terminal when testing.. issue here is that this is TS so not a standard dimos module. we could add an empty hosting module that just runs the TS file

yes, i was thinking about it. would be nice if we auto-start in the blueprint

I think actual QuestTeleopModule should start this and stop it maybe? would this make it easily usable in blueprints? How come there is no example blueprint that works with xarm btw?

That might be one way to do it. If QuestTeleopModule.start() autolaunches the server and if stop() kills it, there's no need for extra terminal, and nothing to mention in blueprint. This can be implemented using subprocess, and can be solved by making few additions in start() and stop().
But, this involves including server related lines in the Module, which doesn't feel very good. Receiving msgs from device via LCM gave us the benefit to keep the module independent of server/ws related code. So, not entirely sure on this.

Anonther pro - This webserver and deno bridge are explicitly for quest. so, just including its lifecycle with the quest module makes sense, rather than another hosting module to connect with quest module.
Its highly unlikely we would use that TS-server-module to connect with a non-quest module.

Reg xarm+quest_teleop blueprint, as mentioned in the below comment - few changes and additions are required in dioms/control/tasks for the teleop to work with arms. So, preparing to make another short PR for this.

In terms of pro/cons - you are designing an interface, think about what other developer "as a user" wants from you. They don't want to know or mange your lifecycle or implementation details, is it deno server? is it .apk using TCP? no one wants to know actually.

Ideally a dev can think only about stuff they care about, so topics you provide for teleop and start/stop, the rest is implementation details. so IMO starting a server (and whatever else you need to provide your interface) is ok but approving the PR for now and you can iterate

ruthwikdasyam · 2026-02-08T23:50:09Z

there is this issue with forwarding all LCM packets to quest otherwise looks good.

Modified - No forwarding to quest in this PR. Will be adding this feature later

Generally for PRs pls include an example how to run this so people can test easily, I'm missing for example an XAarm teleop blueprint or something to make it clear how this is to be used

yes, added test in PR description. We can run a blueprint and see poses and button presses through rerun viz.
Did test with xarm, and that requires few upgrades with the arm controls. I have it ready, will be making a seperate PR for that. Thus, we will have a quest-xarm-teleop blueprint to test.

leshy · 2026-02-09T04:16:28Z

#1222 there is a way to do this without a separate TS server, others can prioritize when/if to do this

ruthwikdasyam · 2026-02-09T05:33:37Z

#1222 there is a way to do this without a separate TS server, others can prioritize when/if to do this

Great. Will look into this, and any other alternatives too. I was trying a similar approach having a weserver on python side and receiving msgs in JSON. But, switched to have it in LCM to maintain DimOS style
Prioritizing and making it a single module will help, as this design will be carried to other teleop-device integrations as well.

leshy

approving with proposed imho better approach #1222

@jca0

Release v0.0.10: Manipulation Stack, MuJoCo Simulation, DDS Transport, Web and Native Visualization via Rerun ## Highlights 88+ commits, 20 contributors, 700+ files changed. The TLDR: **a complete manipulation stack**, **MuJoCo simulation**, **DDS transport**, and **a rewritten visualization pipeline**. Agents are no longer bolted on top — they're refactored as native modules with direct stream access. The entire ROS message dependency has been removed from core DimOS, and we've added VR, phone, and arm teleoperation stacks. You can now vibecode a pick-and-place task from natural language to motor commands. Installation has been significantly streamlined — no more direnv, simpler setup, and the web viewer is now the default. --- ## 🚀 New Features ### Simulation - **MuJoCo simulation module** — Run any DimOS blueprint in simulation with no hardware. Supports xArm and Unitree embodiments, parses MJCF/URDF for robot properties, monotonic clock timing (no `time.sleep`). `dimos --simulation run unitree-go2` ([#1035](#1035)) by @jca0 - **Simulation teleop blueprints** — Added simulation teleop blueprints for Piper, xArm6, and xArm7. ([#1308](#1308)) by @mustafab0 ### Manipulation - **Modular manipulation stack** — Full planning stack with Drake: FK/IK solvers (Jacobian + Drake optimization), RRT path planning, world model with obstacle monitoring, multi-robot management. xArm6/7 and Piper support. ([#1079](#1079)) by @mustafab0 - **Joint servo and cartesian controllers** — Joint position/velocity controllers and cartesian IK task with Pinocchio solver. PoseStamped stream input for real-time control. ([#1116](#1116)) by @mustafab0 - **GraspGen integration** — Grasp generation via Docker-hosted GPU model. Lazy container startup, thread-safe init, RPC `generate_grasps()` returns ranked PoseArray. ([#1119](#1119), [#1234](#1234)) by @JalajShuklaSS - **Gripper control** — Gripper RPC methods on control coordinator, exposed adapter property for custom implementations. ([#1213](#1213)) by @mustafab0 - **Detection3D and Object support** — Object input topics, TF support on manipulation module, pointcloud-to-convex-hull for Drake imports. ([#1236](#1236)) by @mustafab0 - **Agentic pick and place** — Reimplemented manipulation skills for agent-driven pick-and-place workflows. ([#1237](#1237)) by @mustafab0 ### Teleoperation - **Quest VR teleoperation** — Full WebXR + Deno bridge stack. Quest controller data (pose, trigger, grip) streamed to DimOS modules. Monitor-style locking for control loops. ([#1215](#1215)) by @ruthwikdasyam - **Phone teleoperation** — Control Go2 from your phone with a web-based teleop interface. ([#1280](#1280)) by @ruthwikdasyam - **Arm teleop with Pinocchio IK** — Single and dual arm teleoperation using Pinocchio inverse kinematics. Blueprints for xArm, Piper, and dual configurations. ([#1246](#1246)) by @ruthwikdasyam ### Transports & Infrastructure - **DDS transport protocol** — CycloneDDS transport with configurable QoS (high-throughput and reliable profiles). Optional install, benchmark integration. ([#1174](#1174)) by @Kaweees - **Pubsub pattern subscriptions** — Glob and regex pattern matching for topic subscriptions. `subscribe_all()` for bridge-style consumers. Topic type encoding in channel strings (`/topic#module.ClassName`). ([#1114](#1114)) by @leshy - **LCM raw bytes passthrough** — Skip `lcm_encode()` when message is already bytes. ([#1223](#1223)) by @leshy - **Unified TimeSeriesStore** — Pluggable backends (InMemory, SQLite, Pickle, PostgreSQL) with SortedKeyList for O(log n) operations. Replaces the old replay system and TimestampedCollection. Collection API with slice, range, and streaming methods. ([#1080](#1080)) by @leshy - **DimosROS benchmark tests** — Benchmark suite for ROS transport performance. ([#1087](#1087)) by @leshy ### Navigation - **FASTLIO2 support** — Hardware-verified localization with arm64 support. Docker deployment with FAR Planner, terrain analysis, and bagfile playback mode. Builds or-tools from source on arm64. ([#1149](#1149)) by @baishibona - **Native Livox + FASTLIO2 module** — First-class DimOS native module for Livox Mid-360 lidar with FASTLIO2 localization. ([#1235](#1235)) by @leshy ### Visualization - **RerunBridge module and CLI** — New bridge that subscribes to all LCM messages and logs those with `to_rerun()` to Rerun viewer. GlobalConfig singleton, web viewer support. Replaces the old rerun initialization system. ([#1154](#1154)) by @leshy - **Webcam rerun visualization** — Camera module logs to Rerun with pinhole projection for 3D visualization. ([#1117](#1117)) by @ruthwikdasyam - **Default viewer switched to rerun-web** — Browser-based viewer is now the default for broader compatibility. No native viewer install needed. ([#1324](#1324)) by @spomichter ### Agents - **Agent refactor** — Restructured agent module with cleaner imports and global config integration. ([#1211](#1211)) by @paul-nechifor - **Timestamp knowledge** — Agents now have timestamp awareness in prompts for temporal reasoning. ([#1093](#1093)) by @ClaireBookworm - **Observe skill** — Go2 can now observe (capture and describe) its environment via agent skill. ([#1109](#1109)) by @paul-nechifor ### Platform & Hardware - **G1 without ROS** — Unitree G1 blueprints decoupled from ROS dependency. Lazy imports for fast startup. ([#1221](#1221)) by @jeff-hykin - **ARM (aarch64) support** — DimOS runs on ARM hardware. Platform-conditional dependencies, open3d source builds for arm64. ([#1229](#1229)) by @jeff-hykin - **Universal joint/hardware schema** — `HardwareComponent` dataclass with `JointState`, `JointName` type aliases. Backend registry with auto-discovery for SDK adapters. ([#1040](#1040), [#1067](#1067)) by @mustafab0 --- ## 🔧 Improvements - **Optional Dask** — Start without Dask using `--no-dask` flag. Startup time reduced from ~60s to ~45s. ([#1111](#1111), [#1232](#1232)) by @paul-nechifor - **RPC rework** — Renamed `ModuleBlueprint` → `_BlueprintAtom`, `ModuleBlueprintSet` → `Blueprint`, `ModuleConnection` → `Stream`. Added `ModuleRef`, improved type hints throughout. ([#1143](#1143)) by @jeff-hykin - **Image class simplification** — Rewritten as pure NumPy dataclass. Removed CUDA backend, unused methods (solve_pnp, csrt_tracker), and image_impls/ directory. ([#1161](#1161)) by @leshy - **Odometry message cleanup** — Simplified Odometry message type. ([#1256](#1256)) by @leshy - **Remove all ROS message dependencies** — Purged ROS message types from core DimOS. Refactored rosnav to use ROSTransport. Removed dead ROS bridge code. ([#1230](#1230)) by @alexlin2 - **Removed bad function serialization** — Eliminated unnecessary serialization of Python functions. ([#1121](#1121)) by @paul-nechifor - **Benchmark IEC units** — Switched bandwidth benchmarks from SI to IEC units for accuracy. ([#1147](#1147)) by @leshy - **Pubsub typing improvements** — Thread-safety locks on `subscribe_new_topics` and `subscribe_all`. Proper type params across pubsub stack. ([#1153](#1153)) by @leshy - **Autogenerated blueprint list** — Blueprints are now auto-discovered and listed. ([#1100](#1100)) by @paul-nechifor - **Generic Buttons message** — Renamed `QuestButtons` to `Buttons` with generic field names for cross-platform teleop. ([#1261](#1261)) by @ruthwikdasyam - **Dev container uses ros-dev image** — `./bin/dev` now runs the ROS-enabled dev image. ([#1170](#1170)) by @leshy - **LSP support** — Added python-lsp-server and python-lsp-ruff to dev dependencies. ([#1169](#1169)) by @leshy - **Lazy-load pyrealsense2** — RealSense camera module uses lazy imports to avoid errors in simulation environments without the SDK. ([#1309](#1309)) by @spomichter - **Removed unused mmcv and mmengine** — Dead Detic dependencies removed, eliminating slow source builds from install. ([#1319](#1319)) by @spomichter - **Simplified installation** — Removed direnv requirement, streamlined install instructions across all platforms. ([#1315](#1315)) by @spomichter - **DDS extra excluded from --all-extras** — `cyclonedds` requires a source build, so `dds` is now excluded from `uv sync --all-extras` by default. ([#1318](#1318)) by @spomichter - **Nix pre-commit skip** — Skip pre-commit install if hooks already exist. ([#1162](#1162)) by @leshy - **Removed base-requirements** — Consolidated dependency management. ([#1098](#1098)) by @paul-nechifor - **Removed old graspnet** — Cleaned up deprecated graspnet version. ([#1248](#1248)) by @paul-nechifor - **Code cleanup** — Removed `tofix` markers ([#1216](#1216)), fixed ruff issues ([#1112](#1112)), removed old README_installation.md ([#1101](#1101)) by @paul-nechifor --- ## 🐛 Bug Fixes - Fix LFS updating (move from .local to venv) ([#1090](#1090)) by @jeff-hykin - Launch hotfixes: git clone HTTPS, get_data main branch ([#1091](#1091)) by @spomichter - Fix camera demo not showing in Rerun ([#1148](#1148)) by @jeff-hykin - Default to rerun native viewer ([#1099](#1099)) by @Nabla7 - Fix exploration blocking agent loop ([#1258](#1258)) by @paul-nechifor - Fix person-follow blocking agent loop ([#1278](#1278)) by @paul-nechifor - Skip metric3d tests on unsupported xformers GPUs (Blackwell compute capability >9.0) ([#1225](#1225)) by @leshy - Fix manipulation tests ([#1218](#1218), [#1247](#1247)) by @jeff-hykin, @paul-nechifor - Fix control coordinator e2e test ([#1212](#1212)) by @mustafab0 - Fix xarm7-sim broken e2e tests ([#1294](#1294)) by @paul-nechifor - Pin langchain to restore supported providers ([#1241](#1241)) by @spomichter - Fix missing library dependencies in Nix flake ([#1240](#1240)) by @Kaweees - Fix discord invite link ([#1122](#1122)) by @spomichter - macOS edgecase fix ([#1096](#1096)) by @jeff-hykin - Fix second N in logo ([#1250](#1250)) by @jeff-hykin - Fix Unitree Go2 minor issues ([#1307](#1307)) by @paul-nechifor - Fix broken tests ([#1305](#1305)) by @ruthwikdasyam - Fix `uv sync` for some macOS systems ([#1322](#1322)) by @jeff-hykin - Fix mmcv install ([#1313](#1313)) by @paul-nechifor - Fix mypy issues ([#1150](#1150), [#1167](#1167), [#1257](#1257)) by @leshy, @paul-nechifor, @jeff-hykin - Fix Nix install uv pip extras ([#1321](#1321)) by @spomichter --- ## 📚 Documentation - **Major docs overhaul** — New README with feature grid, hardware table, quickstart. Navigation, transports, data streams, and agent docs. ([#1295](#1295)) by @leshy - **Day 1 docs** — Comprehensive getting started guide, development docs, contributing guide, architecture overview. Executable blueprint docs via md-babel-py. ([#1064](#1064)) by @jeff-hykin - **Arm integration guide** — How-to for integrating new robotic arms with DimOS. ([#1238](#1238)) by @mustafab0 - **MCP documentation update** — Updated MCP install and usage instructions. ([#1251](#1251)) by @Kaweees - **Docker docs** — First pass on Docker deployment documentation. ([#1151](#1151)) by @leshy - **Transports documentation** — Encode/decode mixins, SHM examples, ROS/DDS transport docs. ([#1107](#1107)) by @leshy - **Rerun API examples** — Updated examples for the new RerunBridge API. ([#1262](#1262)) by @jeff-hykin - **PR template** added ([#1172](#1172)) by @christiefhyang - **Simplified install instructions** — Removed direnv, streamlined across all platforms. ([#1315](#1315)) by @spomichter - **Python example restored** — Added back the Python usage example. ([#1317](#1317)) by @jeff-hykin - **Nix install updated** — Replaced uv with pip for Nix compatibility. ([#1326](#1326)) by @ruthwikdasyam - **README improvements** ([#1311](#1311)) by @paul-nechifor - **Simplified writing docs** — Consolidated writing_docs to a single markdown file. ([#1254](#1254)) by @jeff-hykin --- ## 🏗️ CI & Build - **ci-complete gate** — Dynamic branch protection via single aggregated status check. MD-only PRs no longer blocked. ([#1279](#1279)) by @spomichter - **Path-based test filtering** — Test jobs fully skip (no container spin-up) when no relevant code changed. ([#1284](#1284), [#1286](#1286)) by @spomichter - **Navigation docker build workflow** — CI builds for the ROS navigation stack. ([#1259](#1259)) by @spomichter - **CUDA test marker** — `@pytest.mark.cuda` for GPU-dependent tests. ([#1220](#1220)) by @jeff-hykin - **e2e test marker** — Marked end-to-end tests for selective CI runs. ([#1110](#1110)) by @paul-nechifor - **pytest stdin fix** — Added `-s` to default addopts for LCM autoconf compatibility. ([#1320](#1320)) by @spomichter --- ## ⚠️ Breaking Changes - **RPC renames**: `ModuleBlueprint` → `_BlueprintAtom`, `ModuleBlueprintSet` → `Blueprint`, `ModuleConnection` → `Stream` ([#1143](#1143)) - **Image class rewrite**: `CudaImage` and `NumpyImage` removed. Image is now a pure NumPy dataclass. Methods like `solve_pnp`, `csrt_tracker`, `from_depth`, `to_depth_meters` removed. ([#1161](#1161)) - **ROS messages removed from core**: All `to_ros`/`from_ros` conversion methods removed. Use `ROSTransport` instead. ([#1230](#1230)) - **QuestButtons → Buttons**: Renamed with generic field names. ([#1261](#1261)) - **RerunBridge replaces old rerun init**: `dimos.dashboard.rerun_init` removed. Use `RerunBridgeModule` or the `rerun-bridge` CLI. ([#1154](#1154)) - **Unitree directory restructuring**: `unitree_go2` → `unitree/go2`, `unitree_g1` → `unitree/g1`. Blueprint names updated. ([#1221](#1221)) - **Default viewer is now rerun-web**: Use `--viewer-backend rerun` to restore native viewer. ([#1324](#1324)) --- ## Quickstart ```bash # Install uv pip install dimos[base,unitree] # Try it (no hardware needed) # NOTE: First run downloads ~2.4 GB from LFS dimos --replay run unitree-go2 # Simulate uv pip install dimos[base,unitree,sim] dimos --simulation run unitree-go2 ``` --- ## New Contributors 🎉 - @ruthwikdasyam — Quest VR teleoperation, phone teleop, arm teleop, webcam rerun viz - @JalajShuklaSS — GraspGen integration - @jca0 — MuJoCo simulation module - @christiefhyang — PR template --- **Full Changelog**: [v0.0.9...v0.0.10](v0.0.9...v0.0.10)

ruthwikdasyam and others added 27 commits January 26, 2026 23:01

Initial Commit

ba4edaa

CI code cleanup

4cea4e9

Feat: adding unit32

877e1e8

Feat: webXR and deno bridge

bfe72d5

Introducing base teleop protocol

179e809

Feat: quest module and its subclasses

0824157

Feat: nice API for data from quest controllers

5844c89

Feat: Quest utils - visualization and transforms

737a1b1

Docs: Readme added

584dfbc

Feat: pose sub method

2cd8b8a

Misc: init updates

4e2e3da

updating teleop blueprints

9538e28

Feat: subclass for arm teleop

2f3fabd

Fix: Bugs and cleanup

19d4e0c

Misc: readme updates

8bc29f3

Fix: pre-commit errors

a7fcc50

Merge origin/dev into ruthwik_teleop

520a87c

Feat: added locks, engage feat changes

9e46122

Misc: renaming methods and docstrings

db8a4df

ignore teleop server certs

e2cb985

Feat: trigger and grip sent through axes to preserve floats

13fffe3

Fix: disposable wrapper

7c61378

Fix: deleted file

459cfad

Merge branch 'dev' into ruthwik_teleop

1b0ecab

Misc: docstrings, comments and mypy fixes

a16e712

Misc: readme changes + added Constants

3401d11

Feat: lock in control loop - monitor-style locking

e94574e

ruthwikdasyam marked this pull request as ready for review February 8, 2026 05:46

greptile-apps bot reviewed Feb 8, 2026

View reviewed changes

ruthwikdasyam added 3 commits February 7, 2026 21:57

Fix: resolve re-entrant lock in control loop and LCM init order

12f036f

Fix: logging malperformed joy msgs

92147a6

Fix: stream logging

84315c4

paul-nechifor reviewed Feb 8, 2026

View reviewed changes

ruthwikdasyam added 2 commits February 8, 2026 01:15

Fix: teleop certs path

1ae2d7e

Fix: server run cmd

eac7ed1

leshy reviewed Feb 8, 2026

View reviewed changes

leshy requested changes Feb 8, 2026

View reviewed changes

leshy reviewed Feb 8, 2026

View reviewed changes

Fix: Not forwarding any msgs to quest, Readme updates

8a37d4d

leshy mentioned this pull request Feb 9, 2026

Quest teleop Single Module #1222

Open

leshy approved these changes Feb 9, 2026

View reviewed changes

leshy merged commit 020105c into dev Feb 9, 2026
15 checks passed

ruthwikdasyam deleted the ruthwik_teleop branch February 19, 2026 02:53

spomichter mentioned this pull request Feb 21, 2026

Release v0.0.10: Manipulation Stack, MuJoCo Simulation, DDS Transport, Web and Native Visualization via Rerun #1336

Merged

		@@ -0,0 +1,95 @@
		#!/usr/bin/env -S deno run --allow-net --allow-read --allow-run --allow-write --unstable-net

Conversation

ruthwikdasyam commented Feb 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Feature

Issue - Linear

Approach/Solution

Web stuff

Architecture

Refer for Detailed Spec

Breaking Changes

How to Test

Files

Uh oh!

greptile-apps bot commented Feb 8, 2026

Greptile Overview

Greptile Summary

Confidence Score: 2/5

Important Files Changed

Sequence Diagram

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot commented Feb 8, 2026

Uh oh!

ruthwikdasyam commented Feb 8, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

leshy left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

leshy Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ruthwikdasyam Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ruthwikdasyam Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ruthwikdasyam commented Feb 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

leshy commented Feb 9, 2026

Uh oh!

ruthwikdasyam commented Feb 9, 2026

Uh oh!

leshy left a comment

Choose a reason for hiding this comment

Uh oh!

ruthwikdasyam commented Feb 7, 2026 •

edited

Loading

leshy Feb 9, 2026 •

edited

Loading

ruthwikdasyam Feb 9, 2026 •

edited

Loading

ruthwikdasyam Feb 9, 2026 •

edited

Loading

ruthwikdasyam commented Feb 8, 2026 •

edited

Loading