Feat: Phone teleoperation stack with Go2 example

[ruthwikdasyam]

Phone Teleoperation for DimOS

Adding teleoperation via smartphone motion sensors. Tilt the phone to drive.
This extends the teleop subsystem introduced in the Quest PR, reusing the same architecture (WebSocket -> Deno LCM bridge -> Python module) but adapted for phone DeviceOrientation/DeviceMotion APIs

Issue - Linear

closes DIM-393
closes DIM-535

Approach

Web layer

• Mobile web app (index.html) reads DeviceOrientation (roll, pitch, yaw) and DeviceMotion (gyro rates) at native sensor frequency
• Encodes sensor data as TwistStamped and button state as Bool, sends over WebSocket at 50 Hz

Python module

• PhoneTeleopModule subscribes to /phone/sensors (TwistStamped) and /phone/button (Bool) LCM channels
• On engage (button hold), captures home orientation. Control loop at 50 Hz computes orientation delta from home, applies configurable gains, publishes TwistStamped velocity commands
• Implements TeleopProtocol - same interface as QuestTeleopModule
• Subclass hooks: _handle_engage(), _should_publish(), _get_output_twist(), _publish_msg()
• Auto-launches/stops the Deno bridge subprocess with proper SIGTERM/SIGKILL shutdown

Also in this PR (teleop modules' improvements)

• Deno bridge auto-launches with the module - no second terminal needed.
• Robust subprocess lifecycle for both phone and quest teleop modules.

Breaking Changes

None

How to Test

With a Unitree Go2:

1. Set ROBOT_IP env var to the Go2's IP
2. Run dimos run phone-go2-teleop
3. On phone, open https://<host-ip>:8444 and accept the self-signed certificate
4. Tap "Start Sensors" - grant orientation/motion permissions
5. Tap "Connect" - WS badge should turn blue
6. Press and hold "HOLD TO TELEOP" - tilt phone to drive the Go2
7. Release button - robot stops
8. Tap "Disconnect"

[greptile-apps]

Greptile Summary

Adds a phone teleoperation stack that lets users drive robots by tilting a smartphone. The architecture mirrors the existing Quest VR teleop: a mobile web app reads DeviceOrientation/DeviceMotion sensor data, encodes it as LCM messages over WebSocket, and a Deno bridge forwards to Python. PhoneTeleopModule captures a home orientation on button press, computes delta-based velocity commands at 50 Hz, and publishes TwistStamped. Subclasses (SimplePhoneTeleop, PhoneGo2Teleop) filter axes for ground robots and wire to the Go2 cmd_vel interface. The PR also adds auto-launch of the Deno bridge subprocess to both phone and Quest modules, and moves Quest blueprints to dimos/teleop/quest/blueprints.py.

• New PhoneTeleopModule implements TeleopProtocol with configurable gains, orientation-delta control, and overridable hooks for subclassing
• SimplePhoneTeleop filters to mobile-base axes (pitch→linear.x, roll→linear.y, yaw→angular.z); PhoneGo2Teleop adds cmd_vel: Out[Twist] for direct Go2 wiring
• Mobile web app handles iOS permission flow, 50 Hz WebSocket send loop, and pointer-event-based hold-to-drive button
• Deno bridge server auto-generates self-signed TLS certs and uses a separate port (8444) from the Quest bridge (8443)
• Both phone and Quest modules now auto-launch/stop the Deno bridge subprocess with SIGTERM/SIGKILL shutdown — however, _stop_server() has an unhandled TimeoutExpired exception after SIGKILL that can leak the process handle
• Twist gains __add__ and __sub__ operators, used by the phone module for orientation delta computation
• Server management code (_start_server/_stop_server) is duplicated between the phone and Quest modules and should be extracted into shared infrastructure

Confidence Score: 4/5

• This PR is safe to merge with minor fixes needed in the subprocess shutdown path
• The overall architecture is well-designed and follows existing patterns from the Quest teleop module. The code is clean, well-documented, and implements the TeleopProtocol correctly. The only functional issue is the unhandled TimeoutExpired exception in _stop_server() after SIGKILL, which could cause process handle leaks and exception propagation during shutdown. The duplicated server management code is a maintainability concern but not a correctness issue.
• dimos/teleop/phone/phone_teleop_module.py and dimos/teleop/quest/quest_teleop_module.py need attention for the unhandled exception in _stop_server()

Important Files Changed

Filename	Overview
dimos/msgs/geometry_msgs/Twist.py	Adds __add__ and __sub__ operators for component-wise Twist arithmetic. Clean implementation consistent with existing Vector3 operators.
dimos/robot/all_blueprints.py	Updates quest blueprint paths from dimos.teleop.blueprints to dimos.teleop.quest.blueprints after the file move, and adds new phone teleop blueprint/module entries.
dimos/teleop/phone/blueprints.py	Defines simple_phone_teleop and phone_go2_teleop blueprint configurations with LCM transport wiring. Clean and follows existing blueprint conventions.
dimos/teleop/phone/phone_extensions.py	Defines SimplePhoneTeleop (mobile-base axis filtering) and PhoneGo2Teleop (cmd_vel output for Go2 autoconnect). Well-structured subclass hierarchy.
dimos/teleop/phone/phone_teleop_module.py	Core phone teleop module with sensor processing, control loop, and Deno bridge management. Has an unhandled TimeoutExpired exception in _stop_server() and duplicated server management code shared with Quest module.
dimos/teleop/phone/web/static/index.html	Mobile web app for phone teleop. Handles DeviceOrientation/Motion APIs, iOS permissions, and WebSocket LCM encoding at 50 Hz. Well-implemented with proper pointer event handling.
dimos/teleop/phone/web/teleop_server.ts	Deno HTTPS/WSS-to-LCM bridge server on port 8444 (separate from Quest's 8443). Auto-generates self-signed certs. Mirrors the Quest server structure closely.
dimos/teleop/quest/blueprints.py	File moved from dimos/teleop/blueprints.py to dimos/teleop/quest/blueprints.py with no content changes. Path references in all_blueprints.py updated accordingly.
dimos/teleop/quest/quest_teleop_module.py	Adds auto-launch/stop of Deno bridge server subprocess. Same TimeoutExpired bug and code duplication issue as the phone module.

Sequence Diagram

sequenceDiagram
    participant Phone as Phone Browser
    participant Deno as Deno Bridge (port 8444)
    participant LCM as LCM Transport
    participant PTM as PhoneTeleopModule
    participant Robot as Robot (Go2)

    Phone->>Phone: DeviceOrientation + DeviceMotion events
    loop Every 20ms (50 Hz)
        Phone->>Deno: WSS: TwistStamped (roll, pitch, yaw + gyro)
        Phone->>Deno: WSS: Bool (button state)
        Deno->>LCM: UDP: /phone/sensors (TwistStamped)
        Deno->>LCM: UDP: /phone/button (Bool)
    end

    LCM->>PTM: Subscribe /phone/sensors
    LCM->>PTM: Subscribe /phone/button

    loop Control Loop (50 Hz)
        PTM->>PTM: _handle_engage() (button hold → capture home)
        PTM->>PTM: _get_output_twist() (delta from home × gains)
        PTM->>PTM: _publish_msg() → cmd_vel: Out[Twist]
        PTM->>Robot: Twist velocity command
    end

Loading

_{Last reviewed commit: beb8fd1}

[greptile-apps]

_{11 files reviewed, 3 comments}

_{Edit Code Review Agent Settings | Greptile}

[spomichter]

nice

Resolved!

[ruthwikdasyam]

Latest Changes:

• removed protocol, no baseclass nor protocol necessary
• no rpc calls, can be added later if/when in need
• As no protocol, simplified PhoneTeleopModule