PRODUCTION Nav skills on drone with tracking

[spomichter]

Add MAVLink Drone Integration with Visual Servoing

Summary

Complete drone integration via RosettaDrone MAVLink bridge with autonomous object tracking capabilities.

Key Features

• MAVLink drone control - Arm, takeoff, land, fly_to GPS coordinates, velocity movement
• Visual servoing - Object tracking using Qwen detection + CSRT tracker + PID controller
• Indoor/Outdoor modes - Conservative indoor PID (1 m/s max) vs aggressive outdoor (5 m/s max)
• Depth estimation - Metric3D integration for 3D perception
• Replay mode - Full test coverage with recorded MAVLink/video data

New Files

dimos/robot/drone/
├── drone.py                           # Main orchestrator
├── connection_module.py               # MAVLink + skills
├── camera_module.py                   # Video + depth
├── drone_tracking_module.py           # Visual servoing
├── drone_visual_servoing_controller.py # PID controller
├── mavlink_connection.py              # MAVLink protocol
├── dji_video_stream.py                # GStreamer capture
└── test_drone.py                      # Unit tests

Other Changes

• Renamed street_map_query → map_query in OSM skill
• Added position marker to OSM map visualization
• Added Foxglove dashboard for drone telemetry

Test Plan

• Unit tests pass (pytest dimos/robot/drone/test_drone.py)
• Manual test with replay mode
• Manual test with real drone (outdoor mode)

[paul-nechifor]

This hangs the RPC call. The default duration is 120 which is also the RPC call timeout. 120+1 means all default calls to track_objects will result in RPC timeouts.

I think you have to change the caller to poll for the result.

There are multiple places where we need to make a long call to an RPC method. @leshy , what do you think about modifying the system to add a way to have async responses to RPC calls. I.e., the initial call ends immediately, and the callee responds back to the caller when it's ready.

[spomichter]

Yeah good idea here. Perhaps some multiprocess / thread handler where we can spawn long processes in RPC calls cleanly

[spomichter]

For now removed blocking

[paul-nechifor]

Did you run the latest code with the real drone?

I've approved the PR, but I think the code is a bit vibe-y and long. For example fly_to in dimos/robot/drone/mavlink_connection.py is very long (200+ lines). A large part of the reason the file is so long is because it contains just 0, on a single line.

I would have abstracted those calls in a separate class. For example, set_position_target_local_ned_send is called several times in the code, but only three arguments are relevant: forward, right, down. Something like:

class MavlinkControl:
    def __init__(self, mavlink): ...

    def body_ned(self, forward, right, down):
        self.mavlink.mav.set_position_target_local_ned_send(
            0,  # time_boot_ms
            self.mavlink.target_system,
            self.mavlink.target_component,
            mavutil.mavlink.MAV_FRAME_BODY_NED,  # Body frame for strafing
            0b0000111111000111,  # type_mask - velocities only, no rotation
            0,
            0,
            0,  # positions (ignored)
            forward,
            right,
            down,  # velocities in m/s
            0,
            0,
            0,  # accelerations (ignored)
            0,
            0,  # yaw, yaw_rate (ignored)
        )
    ...