From d6b5d647c7b9c949bfc27d665e4b151af3add40a Mon Sep 17 00:00:00 2001 From: alexlin2 Date: Sat, 10 May 2025 12:59:27 -0700 Subject: [PATCH 1/2] updated README to be squashed --- README.md | 53 +++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 39 insertions(+), 14 deletions(-) diff --git a/README.md b/README.md index b585d6a4d9..101575187c 100644 --- a/README.md +++ b/README.md @@ -89,7 +89,7 @@ docker compose -f docker/unitree/agents_interface/docker-compose.yml up --build - A Unitree Go2 robot accessible on your network - The robot's IP address -- OpenAI API Key +- OpenAI/Claude/Alibaba API Key ### Python Installation (Ubuntu 22.04) @@ -113,6 +113,23 @@ pip install -r requirements.txt cp default.env .env ``` +### Agent API keys + +Full functionality will require API keys for the following: + +Requirements: +- OpenAI API key (required for OpenAIAgent) +- Claude API key (optional for ClaudeAgent) +- Alibaba API key (optional for QwenAgent) + +These keys can be added to your .env file or exported as environment variables. +``` +export OPENAI_API_KEY= +export CLAUDE_API_KEY= +export ALIBABA_API_KEY= +``` + + ### ROS2 Unitree Go2 SDK Installation #### System Requirements @@ -124,7 +141,7 @@ See [Unitree Go2 ROS2 SDK](https://github.com/abizovnuralem/go2_ros2_sdk) for ad ```bash mkdir -p ros2_ws cd ros2_ws -git clone --recurse-submodules https://github.com/abizovnuralem/go2_ros2_sdk.git src +git clone --recurse-submodules https://github.com/dimensionalOS/go2_ros2_sdk.git src sudo apt install ros-$ROS_DISTRO-image-tools sudo apt install ros-$ROS_DISTRO-vision-msgs @@ -157,7 +174,7 @@ ros2 launch go2_robot_sdk robot.launch.py ```bash # Change path to your Go2 ROS2 SDK installation source /ros2_ws/install/setup.bash -python test/test_planning_agent_web_interface.py +python tests/run.py ``` #### DimOS Interface: @@ -173,17 +190,24 @@ Non-production directories excluded ``` . ├── dimos/ -│ ├── agents/ # Agent implementation and behaviors -│ ├── robot/ # Robot control and hardware interface -│ │ └── unitree/ # Unitree Go2 specific control and skills implementations -│ ├── stream/ # WebRTC and data streaming -│ ├── web/ # DimOS development interface and API -│ │ └── dimos_interface/ # DimOS web interface -│ ├── simulation/ # Robot simulation environments -│ ├── utils/ # Utility functions and helpers -│ └── types/ # Type definitions and interfaces -├── tests/ # Test files -└── docker/ # Docker configuration files and compose definitions +│ ├── agents/ # Agent implementations +│ │ ├── planning/ # Planning-specific agents +│ │ └── execution/ # Execution-specific agents +│ ├── robot/ +│ │ ├── skills/ # AbstractRobotSkill +│ │ ├── control/ # Robot control interfaces +│ │ └── unitree/ # Unitree Go2 specific code +│ │ ├── global_planner/ # Global planners +│ │ └── local_planner/ # Local navigation planners +│ ├── stream/ # Reactive data streams +│ │ ├── video/ # Video streaming +│ │ └── commands/ # Command streaming +│ ├── simulation/ # Simulation environments +│ ├── types/ # Type definitions +│ ├── utils/ # Utility functions +│ └── web/ # Web interface +├── tests/ # Examples +└── docker/ # Docker configs ``` ## Building @@ -434,3 +458,4 @@ Huge thanks to! ## Known Issues - Agent() failure to execute Nav2 action primitives (move, reverse, spinLeft, spinRight) is almost always due to the internal ROS2 collision avoidance, which will sometimes incorrectly display obstacles or be overly sensitive. Look for ```[behavior_server]: Collision Ahead - Exiting DriveOnHeading``` in the ROS logs. Reccomend restarting ROS2 or moving robot from objects to resolve. - ```docker-compose up --build``` does not fully initialize the ROS2 environment due to ```std::bad_alloc``` errors. This will occur during continuous docker development if the ```docker-compose down``` is not run consistently before rebuilding and/or you are on a machine with less RAM, as ROS is very memory intensive. Reccomend running to clear your docker cache/images/containers with ```docker system prune``` and rebuild. + From f8e2923f97c1e23b661b9a8e8b21dff2bc5775b9 Mon Sep 17 00:00:00 2001 From: stash Date: Sun, 18 May 2025 16:22:08 -0700 Subject: [PATCH 2/2] Update README.md --- README.md | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 101575187c..1876868f6c 100644 --- a/README.md +++ b/README.md @@ -35,16 +35,16 @@ We are shipping a first look at the DIMOS x Unitree Go2 integration, allowing fo - Camera feeds (image_raw, compressed_image, etc.) - IMU data - State information -- Lidar / PointCloud primitives 🚧 +- Lidar / PointCloud primitives - Any other generic Unitree ROS2 topics ### Features - **DimOS Agents** - Agent() classes with planning, spatial reasoning, and Robot.Skill() function calling abilities. - - Integrate with any off-the-shelf model: OpenAIAgent, GeminiAgent 🚧, DeepSeekAgent 🚧, HuggingfaceAgent 🚧, etc. + - Integrate with any off-the-shelf hosted or local model: OpenAIAgent, ClaudeAgent, GeminiAgent 🚧, DeepSeekAgent 🚧, HuggingFaceRemoteAgent, HuggingFaceLocalAgent, etc. - Modular agent architecture for easy extensibility and chaining of Agent output --> Subagents input. - - Agent spatial / language memory for location grounded reasoning and recall. 🚧 + - Agent spatial / language memory for location grounded reasoning and recall. - **DimOS Infrastructure** - A reactive data streaming architecture using RxPY to manage real-time video (or other sensor input), outbound commands, and inbound robot state between the DimOS interface, Agents, and ROS2. @@ -69,6 +69,8 @@ We are shipping a first look at the DIMOS x Unitree Go2 integration, allowing fo Configure your environment variables in `.env` ```bash OPENAI_API_KEY= +ALIBABA_API_KEY= +ANTHROPIC_API_KEY= ROBOT_IP= CONN_TYPE=webrtc WEBRTC_SERVER_HOST=0.0.0.0 @@ -106,6 +108,9 @@ source venv/bin/activate sudo apt install portaudio19-dev python3-pyaudio +# Install torch and torchvision if not already installed +pip install -r base-requirements.txt + # Install dependencies pip install -r requirements.txt @@ -118,9 +123,9 @@ cp default.env .env Full functionality will require API keys for the following: Requirements: -- OpenAI API key (required for OpenAIAgent) -- Claude API key (optional for ClaudeAgent) -- Alibaba API key (optional for QwenAgent) +- OpenAI API key (required for all LLMAgents due to OpenAIEmbeddings) +- Claude API key (required for ClaudeAgent) +- Alibaba API key (required for Navigation skills) These keys can be added to your .env file or exported as environment variables. ``` @@ -129,14 +134,13 @@ export CLAUDE_API_KEY= export ALIBABA_API_KEY= ``` - ### ROS2 Unitree Go2 SDK Installation #### System Requirements - Ubuntu 22.04 - ROS2 Distros: Iron, Humble, Rolling -See [Unitree Go2 ROS2 SDK](https://github.com/abizovnuralem/go2_ros2_sdk) for additional installation instructions. +See [Unitree Go2 ROS2 SDK](https://github.com/dimensionalOS/go2_ros2_sdk) for additional installation instructions. ```bash mkdir -p ros2_ws