Skip to content

fix: G1 agent RPC wiring and system prompt#1518

Merged
spomichter merged 8 commits intodevfrom
summer/fix/g1-sim-agent
Mar 11, 2026
Merged

fix: G1 agent RPC wiring and system prompt#1518
spomichter merged 8 commits intodevfrom
summer/fix/g1-sim-agent

Conversation

@SUMMERxYANG
Copy link
Contributor

Problem

Two issues with the G1 agentic sim:

  1. Skill container references G1Connection.move but the sim blueprint deploys G1SimConnection — RPC wiring silently fails, so move/gesture commands error at runtime.
  2. Agent uses the default Go2 system prompt, so it identifies as a quadruped and doesn't know about G1-specific skills (arm gestures, mode commands).

Solution

  1. Added G1ConnectionBase ABC with move and publish_request. Both G1Connection and G1SimConnection now extend it. Skill container references G1ConnectionBase so the blueprint MRO-based wiring resolves regardless of which connection is deployed.
  2. Added G1-specific system prompt with correct skill documentation (move params, arm gestures, mode commands). Follows existing pattern used by drone and manipulation blueprints.

Breaking Changes

None

How to Test

  1. dimos --simulation run unitree-g1-agentic-sim
  2. In another terminal: dimos humancli
  3. Say "hi" — agent should identify as a G1 humanoid, not a Go2 quadruped
  4. Ask it to "walk forward" — should call move(x=0.5, duration=...) without RPC errors
  5. Ask it to "wave hello" — should call execute_arm_command("HighWave")

Contributor License Agreement

  • I have read and approved the CLA.

- Add G1ConnectionBase ABC so skill container RPC calls resolve for both
  G1Connection (hardware) and G1SimConnection (simulation)
- Add G1-specific system prompt describing the humanoid's actual skills
  (move, arm gestures, mode commands)
@greptile-apps
Copy link
Contributor

greptile-apps bot commented Mar 10, 2026

Greptile Summary

This PR fixes two runtime bugs in the G1 agentic simulation: it introduces G1ConnectionBase as an ABC so the skill container's RPC wiring resolves correctly regardless of whether G1Connection or G1SimConnection is deployed, and it adds a G1-specific system prompt so the agent correctly identifies itself as a humanoid and knows its available skills.

Key changes:

  • G1ConnectionBase ABC added to connection.py with abstract @rpc methods for move and publish_request; both G1Connection and G1SimConnection now inherit from it
  • UnitreeG1SkillContainer.rpc_calls updated from G1Connection.*G1ConnectionBase.* to fix silent RPC resolution failures when running with the sim connection
  • New G1_SYSTEM_PROMPT documents move parameters, arm gestures, mode commands, and navigation conventions
  • _agentic_skills blueprint now passes G1_SYSTEM_PROMPT to the agent() factory
  • Three GitHub issue templates added (bug report, feature request, config)

Issues found:

  • speak_skill() is not wired into _agentic_skills, but the new system prompt explicitly instructs the agent to call speak for all verbal communication — this will cause the agent to attempt to use an unavailable tool. The Go2 agentic blueprint (_common_agentic.py) correctly includes speak_skill(); the G1 stack should follow the same pattern.
  • G1ConnectionBase is omitted from __all__ in connection.py despite being the primary public abstraction that external modules now depend on.

Confidence Score: 3/5

  • The RPC wiring fix is correct and safe, but the missing speak_skill registration creates a functional regression for the system prompt's stated communication pattern.
  • The core fix (ABC + RPC wiring) is well-structured and the MRO-based resolution approach is sound. However, the newly introduced system prompt instructs the agent to use speak as its primary communication tool, and that skill is not included in _agentic_skills. This means the agent will attempt to call an unregistered tool every time it tries to respond verbally, which directly undermines the user-facing test scenario described in the PR ("Say 'hi' — agent should identify as a G1 humanoid").
  • dimos/robot/unitree/g1/blueprints/agentic/_agentic_skills.py — missing speak_skill() registration.

Important Files Changed

Filename Overview
dimos/robot/unitree/g1/connection.py Introduces G1ConnectionBase ABC with abstract @rpc methods for move and publish_request; G1Connection extends it. G1ConnectionBase is not exported in __all__.
dimos/robot/unitree/g1/sim.py G1SimConnection now extends G1ConnectionBase instead of Module directly; Module import removed (inherited via base). Clean fix that enables MRO-based RPC wiring for the sim path.
dimos/robot/unitree/g1/skill_container.py RPC call references updated from G1Connection.* to G1ConnectionBase.*, correctly decoupling the skill container from the concrete connection type.
dimos/robot/unitree/g1/blueprints/agentic/_agentic_skills.py Wires G1_SYSTEM_PROMPT into the agent; however, speak_skill() is not included in the blueprint despite the system prompt explicitly instructing the agent to use speak.
dimos/robot/unitree/g1/system_prompt.py New G1-specific system prompt correctly documents movement, arm gesture, mode, and navigation skills; references speak which is not currently wired into the agentic stack.

Class Diagram

%%{init: {'theme': 'neutral'}}%%
classDiagram
    class Module {
        <<framework>>
    }
    class ABC {
        <<abstract>>
    }
    class G1ConnectionBase {
        <<abstract>>
        +move(twist, duration) None
        +publish_request(topic, data) dict
    }
    class G1Connection {
        +ip: str
        +connection_type: str
        +move(twist, duration) None
        +publish_request(topic, data) dict
        +start() None
        +stop() None
    }
    class G1SimConnection {
        +ip: str
        +move(twist, duration) None
        +publish_request(topic, data) dict
        +start() None
        +stop() None
    }
    class UnitreeG1SkillContainer {
        +rpc_calls: ["G1ConnectionBase.move", "G1ConnectionBase.publish_request"]
        +move(x, y, yaw, duration) str
        +execute_arm_command(command_name) str
        +execute_mode_command(command_name) str
    }

    Module <|-- G1ConnectionBase
    ABC <|-- G1ConnectionBase
    G1ConnectionBase <|-- G1Connection
    G1ConnectionBase <|-- G1SimConnection
    UnitreeG1SkillContainer ..> G1ConnectionBase : RPC wiring (MRO-resolved)
Loading

Comments Outside Diff (1)

  1. dimos/robot/unitree/g1/connection.py, line 126 (link)

    G1ConnectionBase missing from __all__

    G1ConnectionBase is now the public contract that both G1SimConnection (in sim.py) and UnitreeG1SkillContainer (in skill_container.py) depend on for wiring. Omitting it from __all__ makes it invisible to consumers performing wildcard imports and understates its importance as the public API surface.

Last reviewed commit: 133e182

SUMMERxYANG and others added 5 commits March 10, 2026 17:04
Add speak_skill() to G1 agentic skills so the agent can use the speak
tool referenced in the system prompt. Export G1ConnectionBase in __all__
since it is the public contract for sim and skill wiring.
The test_module_has_start_and_stop test requires all Module subclasses
to declare start and stop methods. Add them as abstract methods on the
base class to satisfy the test and enforce the contract on subclasses.
Give abstract start()/stop() methods a non-trivial body that chains
to Module via super(), fixing 4 safe-super mypy errors.
@spomichter spomichter merged commit 3bb28f0 into dev Mar 11, 2026
11 checks passed
@spomichter spomichter mentioned this pull request Mar 11, 2026
1 task
spomichter added a commit that referenced this pull request Mar 12, 2026
Release v0.0.11

82 PRs, 10 contributors, 396 files changed.

This release brings a production CLI, MCP tooling, temporal memory, and first-class support for coding agents. Dask has been removed. The entire stack now runs from `dimos run` through `dimos stop`.

### Agent-Native Development

DimOS is now built to be driven by coding agents. Point OpenClaw, Claude Code, or Cursor at [AGENTS.md](AGENTS.md) and they can build, run, and debug Dimensional applications using the CLI and MCP interfaces directly.

- **AGENTS.md** — comprehensive onboarding doc: architecture, CLI reference, skill rules, blueprint quick-reference. Your agent reads this and starts coding.
- **MCP server** — all `@skill` methods exposed as HTTP tools. External agents call `dimos mcp call relative_move --arg forward=0.5` or connect via JSON-RPC.
- **MCP CLI** — `dimos mcp list-tools`, `dimos mcp call`, `dimos mcp status`, `dimos mcp modules`
- **Agent context logging** — MCP tool calls and agent messages logged to per-run JSONL for debugging and replay.

### CLI & Daemon

Full process lifecycle — no more Ctrl-C in tmux.

- `dimos run --daemon` — background execution with health checks and run registry
- `dimos stop [--force]` — graceful shutdown with SIGTERM → SIGKILL fallback
- `dimos restart` — replays the original CLI arguments
- `dimos status` — PID, blueprint, uptime, MCP port
- `dimos log -f` — structured per-run logs with follow, JSON output, filtering
- `dimos show-config` — resolved GlobalConfig with source tracing

### Temporal-Spatial Memory

Robots in physical space ingest hours of video and lidar. Temporal-spatial memory gives them a human-like understanding of the world — causal object relationships, entity tracking through time and physical space, and the ability to answer complex temporal queries:

*Who spends the most time in the kitchen? What time on average do I wake up? Which set of switches toggles the main lights? Who was at the office at 9am last Thursday?*

Traditional frame-level embeddings (CLIP, ViT) lose temporal context and don't scale beyond a handful of frames. Video transformers are expensive and don't operate in RGB-D. Dimensional agents work with video + lidar natively, tracking entities across hours and days.

```bash
dimos --replay --replay-dir unitree_go2_office_walk2 run unitree-go2-temporal-memory
```

### Interactive Viewer

Custom Rerun fork (`dimos-viewer`) is now the default. Click-to-navigate: click a point in the 3D view → PointStamped → A* planner → robot moves.

- Camera | 3D split layout on Go2, G1, and drone blueprints
- Native keyboard teleop in the viewer
- `--viewer rerun|rerun-web|rerun-connect|foxglove|none`

### Drone Support

Drone blueprints modernized to match Go2 composition pattern. `drone-basic` and `drone-agentic` work with replay, Rerun, and the full CLI.

```bash
dimos --replay run drone-basic
dimos --replay run drone-agentic
```

### More

- **Go2 fleet control** — multi-robot with `--robot-ips` (#1487)
- **Replay `--replay-dir`** — select dataset, loops by default (#1519, #1494)
- **Interactive install** — `curl -fsSL .../install.sh | bash` (#1395)
- **Nix on non-Debian Linux** (#1472)
- **Remove Dask** — native worker pool (#1365)
- **Remove asyncio dependency** (#1367)
- **Perceive loop** — continuous observation module for agents (#1411)
- **Worker resource monitor** — `dtop` TUI (#1378)
- **G1 agent wiring fix** (#1518)
- **Rerun rate limiting** — prevents viewer OOM on continuous streams (#1509, #1521)
- **RotatingFileHandler** — prevents unbounded log growth (#1492)
- **Test coverage** (#1397), draft PR CI skip (#1398), manipulation test fixes (#1522)

### Breaking Changes

- `--viewer-backend` renamed to `--viewer`
- Dask removed — blueprints using Dask workers need migration to native worker pool
- Default viewer changed from `rerun-web` to `rerun` (native dimos-viewer)

### Contributors

@spomichter, @PaulNechifor, @ruthwikdasyam, @summeryang, @MustafaBhadsorawala, @leshy, @sambull, @JeffHykin, @RadientBrain

## Contributor License Agreement

- [x] I have read and approved the [CLA](https://github.com/dimensionalOS/dimos/blob/main/CLA.md).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants