feat: FASTLIO2 support with arm64 and hardware-verified localization#1149
Merged
spomichter merged 20 commits intodevfrom Feb 14, 2026
Merged
feat: FASTLIO2 support with arm64 and hardware-verified localization#1149spomichter merged 20 commits intodevfrom
spomichter merged 20 commits intodevfrom
Conversation
Verified working: ./start.sh --hardware --tag humble-fastlio --route-planner - Add docker-compose.deploy.yml for pre-built Docker Hub images - Update build.sh with --fastlio/--arise SLAM type selection - Update start.sh with --tag, --route-planner options - Add FASTLIO2 autonomy bridge, terrain analysis, FAR planner launches - Support for Unitree robot via USE_UNITREE env var
- build.sh now exports IMAGE_TAG="${ROS_DISTRO}-${SLAM_TYPE}"
- start.sh uses local dimos_autonomy_stack:${IMAGE_TAG} image
- docker-compose.yml uses IMAGE_TAG for both simulation and hardware
- Removed Docker Hub pull, prompts user to build if image missing
- Dockerfile builds both SLAM systems in single image - LOCALIZATION_METHOD env var selects arise_slam (default) or fastlio - IMAGE_TAG simplified to just ROS_DISTRO (e.g., humble, jazzy) - build.sh uses fastlio2 branch which has both SLAM systems - start.sh --arise/--fastlio flags set LOCALIZATION_METHOD
The $SLAM_ARG variable wasn't expanding correctly in docker-compose command. Fixed by directly embedding the launch argument based on LOCALIZATION_METHOD env var.
- Add --bagfile flag to start.sh for bagfile playback (use_sim_time=true) - Add dimos_bagfile service to docker-compose.yml - Add MAP_PATH env var and ./maps volume mount for localization mode - Support --fastlio, --route-planner, --rviz flags in bagfile mode
ros-base image doesn't include rviz2, causing build failure for teleop_rviz_plugin and other rviz-dependent packages. Added conditional install that only runs on arm64, with ARG placement optimized for layer caching.
The tare_planner package includes pre-built or-tools binaries for x86_64. On arm64, these fail to link with 'file in wrong format' error. Added conditional build steps for arm64: - Build or-tools v9.8 from source with BUILD_DEPS=ON - Replace x86_64 libraries/headers with arm64 build - Disabled SCIP and COINOR (not needed for routing/TSP solver)
Launch files reference joy_node for joystick input but the joy package was not installed in the runtime stage, causing launch failure.
open3d 0.18.0 has aarch64 wheels for Python 3.8-3.11 (Humble works). open3d 0.19.0 dropped aarch64 Linux wheels entirely. No version has aarch64 wheels for Python 3.12 (Jazzy). Pin to 0.18.0 and build from source only when arm64 + Python >= 3.12.
open3d==0.18.0 was removed from PyPI; only 0.19.0 is available for x86_64 / Python 3.12, breaking the Docker build on amd64. Remove the version pin so pip resolves the latest compatible release. open3d version handling per platform: - amd64 (humble/jazzy): installs from PyPI (currently 0.19.0) - arm64 + jazzy (Python 3.12+): built from source (v0.18.0) in Dockerfile, satisfies the unversioned requirement - arm64 + humble (Python 3.10): installs from PyPI (aarch64 wheels available for older Python)
GLFW requires Xinerama, Xcursor, Xrandr, Xi headers even when BUILD_GUI=OFF. Only affects arm64 + Python 3.12+ builds.
…open3d build - Use Open3D's own dependency installer for completeness - Cap make jobs at 4 to avoid OOM on arm64 - Disable WebRTC module (not needed)
Contributor
Greptile OverviewGreptile SummaryThis PR adds comprehensive FASTLIO2 support alongside the existing arise_slam, enabling runtime SLAM method selection via Major Changes:
Key Implementation Details:
Confidence Score: 4/5
Important Files Changed
Sequence DiagramsequenceDiagram
participant User
participant build.sh
participant start.sh
participant Docker
participant Container
participant ROS2
Note over User,ROS2: Build Phase
User->>build.sh: ./build.sh --jazzy
build.sh->>build.sh: Clone/update ros-navigation-autonomy-stack (fastlio2 branch)
build.sh->>Docker: docker compose build
Docker->>Docker: Build multi-arch image (amd64/arm64)
alt arm64
Docker->>Docker: Build open3d from source
Docker->>Docker: Build or-tools from source
end
Docker->>Docker: Build arise_slam + FASTLIO2
Docker->>Docker: Copy configs for both SLAM methods
Docker-->>build.sh: Image: dimos_autonomy_stack:jazzy
Note over User,ROS2: Runtime Phase (Hardware Mode)
User->>start.sh: ./start.sh --hardware --localization fastlio --route-planner
start.sh->>start.sh: Set LOCALIZATION_METHOD=fastlio
start.sh->>start.sh: Validate .env file exists
start.sh->>start.sh: Check lidar configuration
start.sh->>start.sh: Verify image exists
start.sh->>Docker: docker compose --profile hardware up
Docker->>Container: Start with LOCALIZATION_METHOD env
Container->>Container: source ROS workspace
alt LOCALIZATION_METHOD=fastlio
Container->>ROS2: ros2 launch system_real_robot_with_route_planner.launch.py use_fastlio2:=true
ROS2->>ROS2: Start FASTLIO2 SLAM
else LOCALIZATION_METHOD=arise_slam
Container->>ROS2: ros2 launch system_real_robot_with_route_planner.launch.py
ROS2->>ROS2: Start arise_slam
end
Container->>ROS2: Start route planner (FAR planner)
alt USE_RVIZ=true
Container->>ROS2: ros2 run rviz2 rviz2
end
Container->>ROS2: Start Foxglove bridge
ROS2-->>User: Navigation system ready
Note over User,ROS2: Bagfile Playback Mode
User->>start.sh: ./start.sh --bagfile --localization fastlio
start.sh->>Docker: docker compose --profile bagfile up
Docker->>Container: Start with use_sim_time=true
Container->>ROS2: ros2 launch system_bagfile.launch.py use_fastlio2:=true
User->>Container: ros2 bag play --clock bagfile
ROS2->>ROS2: Process bagfile with FASTLIO2
|
Collaborator
Author
|
DO NOT MERGE UNTIL VERIFIED ON G1 (JETSON) |
- Replace --humble/--jazzy with --image <distro> - Replace --arise/--fastlio with --localization <method> - Remove redundant --tag flag - Group help text by mutually exclusive options - Cap or-tools build parallelism to 4 (match open3d/ceres) - Abort build if local changes in autonomy stack (instead of git reset --hard) - Remove redundant git remote add in build.sh - Update README with new CLI syntax
…onalOS Update build.sh to clone from dimensionalOS/ros-navigation-autonomy-stack instead of VectorRobotics/vector_navigation_stack. Remove redundant vector remote setup.
Collaborator
Author
|
@greptile, @greptile-app. Update to address comments. Please review. |
Contributor
|
@ruthwikdasyam TODO test this on the G1 onboard the jetson NX. @SUMMERxYANG add to linear. is there some way to tag /linear in github comments like this and make issues? would be super helpful |
spomichter
approved these changes
Feb 14, 2026
spomichter
added a commit
that referenced
this pull request
Feb 21, 2026
Release v0.0.10: Manipulation Stack, MuJoCo Simulation, DDS Transport, Web and Native Visualization via Rerun ## Highlights 88+ commits, 20 contributors, 700+ files changed. The TLDR: **a complete manipulation stack**, **MuJoCo simulation**, **DDS transport**, and **a rewritten visualization pipeline**. Agents are no longer bolted on top — they're refactored as native modules with direct stream access. The entire ROS message dependency has been removed from core DimOS, and we've added VR, phone, and arm teleoperation stacks. You can now vibecode a pick-and-place task from natural language to motor commands. Installation has been significantly streamlined — no more direnv, simpler setup, and the web viewer is now the default. --- ## 🚀 New Features ### Simulation - **MuJoCo simulation module** — Run any DimOS blueprint in simulation with no hardware. Supports xArm and Unitree embodiments, parses MJCF/URDF for robot properties, monotonic clock timing (no `time.sleep`). `dimos --simulation run unitree-go2` ([#1035](#1035)) by @jca0 - **Simulation teleop blueprints** — Added simulation teleop blueprints for Piper, xArm6, and xArm7. ([#1308](#1308)) by @mustafab0 ### Manipulation - **Modular manipulation stack** — Full planning stack with Drake: FK/IK solvers (Jacobian + Drake optimization), RRT path planning, world model with obstacle monitoring, multi-robot management. xArm6/7 and Piper support. ([#1079](#1079)) by @mustafab0 - **Joint servo and cartesian controllers** — Joint position/velocity controllers and cartesian IK task with Pinocchio solver. PoseStamped stream input for real-time control. ([#1116](#1116)) by @mustafab0 - **GraspGen integration** — Grasp generation via Docker-hosted GPU model. Lazy container startup, thread-safe init, RPC `generate_grasps()` returns ranked PoseArray. ([#1119](#1119), [#1234](#1234)) by @JalajShuklaSS - **Gripper control** — Gripper RPC methods on control coordinator, exposed adapter property for custom implementations. ([#1213](#1213)) by @mustafab0 - **Detection3D and Object support** — Object input topics, TF support on manipulation module, pointcloud-to-convex-hull for Drake imports. ([#1236](#1236)) by @mustafab0 - **Agentic pick and place** — Reimplemented manipulation skills for agent-driven pick-and-place workflows. ([#1237](#1237)) by @mustafab0 ### Teleoperation - **Quest VR teleoperation** — Full WebXR + Deno bridge stack. Quest controller data (pose, trigger, grip) streamed to DimOS modules. Monitor-style locking for control loops. ([#1215](#1215)) by @ruthwikdasyam - **Phone teleoperation** — Control Go2 from your phone with a web-based teleop interface. ([#1280](#1280)) by @ruthwikdasyam - **Arm teleop with Pinocchio IK** — Single and dual arm teleoperation using Pinocchio inverse kinematics. Blueprints for xArm, Piper, and dual configurations. ([#1246](#1246)) by @ruthwikdasyam ### Transports & Infrastructure - **DDS transport protocol** — CycloneDDS transport with configurable QoS (high-throughput and reliable profiles). Optional install, benchmark integration. ([#1174](#1174)) by @Kaweees - **Pubsub pattern subscriptions** — Glob and regex pattern matching for topic subscriptions. `subscribe_all()` for bridge-style consumers. Topic type encoding in channel strings (`/topic#module.ClassName`). ([#1114](#1114)) by @leshy - **LCM raw bytes passthrough** — Skip `lcm_encode()` when message is already bytes. ([#1223](#1223)) by @leshy - **Unified TimeSeriesStore** — Pluggable backends (InMemory, SQLite, Pickle, PostgreSQL) with SortedKeyList for O(log n) operations. Replaces the old replay system and TimestampedCollection. Collection API with slice, range, and streaming methods. ([#1080](#1080)) by @leshy - **DimosROS benchmark tests** — Benchmark suite for ROS transport performance. ([#1087](#1087)) by @leshy ### Navigation - **FASTLIO2 support** — Hardware-verified localization with arm64 support. Docker deployment with FAR Planner, terrain analysis, and bagfile playback mode. Builds or-tools from source on arm64. ([#1149](#1149)) by @baishibona - **Native Livox + FASTLIO2 module** — First-class DimOS native module for Livox Mid-360 lidar with FASTLIO2 localization. ([#1235](#1235)) by @leshy ### Visualization - **RerunBridge module and CLI** — New bridge that subscribes to all LCM messages and logs those with `to_rerun()` to Rerun viewer. GlobalConfig singleton, web viewer support. Replaces the old rerun initialization system. ([#1154](#1154)) by @leshy - **Webcam rerun visualization** — Camera module logs to Rerun with pinhole projection for 3D visualization. ([#1117](#1117)) by @ruthwikdasyam - **Default viewer switched to rerun-web** — Browser-based viewer is now the default for broader compatibility. No native viewer install needed. ([#1324](#1324)) by @spomichter ### Agents - **Agent refactor** — Restructured agent module with cleaner imports and global config integration. ([#1211](#1211)) by @paul-nechifor - **Timestamp knowledge** — Agents now have timestamp awareness in prompts for temporal reasoning. ([#1093](#1093)) by @ClaireBookworm - **Observe skill** — Go2 can now observe (capture and describe) its environment via agent skill. ([#1109](#1109)) by @paul-nechifor ### Platform & Hardware - **G1 without ROS** — Unitree G1 blueprints decoupled from ROS dependency. Lazy imports for fast startup. ([#1221](#1221)) by @jeff-hykin - **ARM (aarch64) support** — DimOS runs on ARM hardware. Platform-conditional dependencies, open3d source builds for arm64. ([#1229](#1229)) by @jeff-hykin - **Universal joint/hardware schema** — `HardwareComponent` dataclass with `JointState`, `JointName` type aliases. Backend registry with auto-discovery for SDK adapters. ([#1040](#1040), [#1067](#1067)) by @mustafab0 --- ## 🔧 Improvements - **Optional Dask** — Start without Dask using `--no-dask` flag. Startup time reduced from ~60s to ~45s. ([#1111](#1111), [#1232](#1232)) by @paul-nechifor - **RPC rework** — Renamed `ModuleBlueprint` → `_BlueprintAtom`, `ModuleBlueprintSet` → `Blueprint`, `ModuleConnection` → `Stream`. Added `ModuleRef`, improved type hints throughout. ([#1143](#1143)) by @jeff-hykin - **Image class simplification** — Rewritten as pure NumPy dataclass. Removed CUDA backend, unused methods (solve_pnp, csrt_tracker), and image_impls/ directory. ([#1161](#1161)) by @leshy - **Odometry message cleanup** — Simplified Odometry message type. ([#1256](#1256)) by @leshy - **Remove all ROS message dependencies** — Purged ROS message types from core DimOS. Refactored rosnav to use ROSTransport. Removed dead ROS bridge code. ([#1230](#1230)) by @alexlin2 - **Removed bad function serialization** — Eliminated unnecessary serialization of Python functions. ([#1121](#1121)) by @paul-nechifor - **Benchmark IEC units** — Switched bandwidth benchmarks from SI to IEC units for accuracy. ([#1147](#1147)) by @leshy - **Pubsub typing improvements** — Thread-safety locks on `subscribe_new_topics` and `subscribe_all`. Proper type params across pubsub stack. ([#1153](#1153)) by @leshy - **Autogenerated blueprint list** — Blueprints are now auto-discovered and listed. ([#1100](#1100)) by @paul-nechifor - **Generic Buttons message** — Renamed `QuestButtons` to `Buttons` with generic field names for cross-platform teleop. ([#1261](#1261)) by @ruthwikdasyam - **Dev container uses ros-dev image** — `./bin/dev` now runs the ROS-enabled dev image. ([#1170](#1170)) by @leshy - **LSP support** — Added python-lsp-server and python-lsp-ruff to dev dependencies. ([#1169](#1169)) by @leshy - **Lazy-load pyrealsense2** — RealSense camera module uses lazy imports to avoid errors in simulation environments without the SDK. ([#1309](#1309)) by @spomichter - **Removed unused mmcv and mmengine** — Dead Detic dependencies removed, eliminating slow source builds from install. ([#1319](#1319)) by @spomichter - **Simplified installation** — Removed direnv requirement, streamlined install instructions across all platforms. ([#1315](#1315)) by @spomichter - **DDS extra excluded from --all-extras** — `cyclonedds` requires a source build, so `dds` is now excluded from `uv sync --all-extras` by default. ([#1318](#1318)) by @spomichter - **Nix pre-commit skip** — Skip pre-commit install if hooks already exist. ([#1162](#1162)) by @leshy - **Removed base-requirements** — Consolidated dependency management. ([#1098](#1098)) by @paul-nechifor - **Removed old graspnet** — Cleaned up deprecated graspnet version. ([#1248](#1248)) by @paul-nechifor - **Code cleanup** — Removed `tofix` markers ([#1216](#1216)), fixed ruff issues ([#1112](#1112)), removed old README_installation.md ([#1101](#1101)) by @paul-nechifor --- ## 🐛 Bug Fixes - Fix LFS updating (move from .local to venv) ([#1090](#1090)) by @jeff-hykin - Launch hotfixes: git clone HTTPS, get_data main branch ([#1091](#1091)) by @spomichter - Fix camera demo not showing in Rerun ([#1148](#1148)) by @jeff-hykin - Default to rerun native viewer ([#1099](#1099)) by @Nabla7 - Fix exploration blocking agent loop ([#1258](#1258)) by @paul-nechifor - Fix person-follow blocking agent loop ([#1278](#1278)) by @paul-nechifor - Skip metric3d tests on unsupported xformers GPUs (Blackwell compute capability >9.0) ([#1225](#1225)) by @leshy - Fix manipulation tests ([#1218](#1218), [#1247](#1247)) by @jeff-hykin, @paul-nechifor - Fix control coordinator e2e test ([#1212](#1212)) by @mustafab0 - Fix xarm7-sim broken e2e tests ([#1294](#1294)) by @paul-nechifor - Pin langchain to restore supported providers ([#1241](#1241)) by @spomichter - Fix missing library dependencies in Nix flake ([#1240](#1240)) by @Kaweees - Fix discord invite link ([#1122](#1122)) by @spomichter - macOS edgecase fix ([#1096](#1096)) by @jeff-hykin - Fix second N in logo ([#1250](#1250)) by @jeff-hykin - Fix Unitree Go2 minor issues ([#1307](#1307)) by @paul-nechifor - Fix broken tests ([#1305](#1305)) by @ruthwikdasyam - Fix `uv sync` for some macOS systems ([#1322](#1322)) by @jeff-hykin - Fix mmcv install ([#1313](#1313)) by @paul-nechifor - Fix mypy issues ([#1150](#1150), [#1167](#1167), [#1257](#1257)) by @leshy, @paul-nechifor, @jeff-hykin - Fix Nix install uv pip extras ([#1321](#1321)) by @spomichter --- ## 📚 Documentation - **Major docs overhaul** — New README with feature grid, hardware table, quickstart. Navigation, transports, data streams, and agent docs. ([#1295](#1295)) by @leshy - **Day 1 docs** — Comprehensive getting started guide, development docs, contributing guide, architecture overview. Executable blueprint docs via md-babel-py. ([#1064](#1064)) by @jeff-hykin - **Arm integration guide** — How-to for integrating new robotic arms with DimOS. ([#1238](#1238)) by @mustafab0 - **MCP documentation update** — Updated MCP install and usage instructions. ([#1251](#1251)) by @Kaweees - **Docker docs** — First pass on Docker deployment documentation. ([#1151](#1151)) by @leshy - **Transports documentation** — Encode/decode mixins, SHM examples, ROS/DDS transport docs. ([#1107](#1107)) by @leshy - **Rerun API examples** — Updated examples for the new RerunBridge API. ([#1262](#1262)) by @jeff-hykin - **PR template** added ([#1172](#1172)) by @christiefhyang - **Simplified install instructions** — Removed direnv, streamlined across all platforms. ([#1315](#1315)) by @spomichter - **Python example restored** — Added back the Python usage example. ([#1317](#1317)) by @jeff-hykin - **Nix install updated** — Replaced uv with pip for Nix compatibility. ([#1326](#1326)) by @ruthwikdasyam - **README improvements** ([#1311](#1311)) by @paul-nechifor - **Simplified writing docs** — Consolidated writing_docs to a single markdown file. ([#1254](#1254)) by @jeff-hykin --- ## 🏗️ CI & Build - **ci-complete gate** — Dynamic branch protection via single aggregated status check. MD-only PRs no longer blocked. ([#1279](#1279)) by @spomichter - **Path-based test filtering** — Test jobs fully skip (no container spin-up) when no relevant code changed. ([#1284](#1284), [#1286](#1286)) by @spomichter - **Navigation docker build workflow** — CI builds for the ROS navigation stack. ([#1259](#1259)) by @spomichter - **CUDA test marker** — `@pytest.mark.cuda` for GPU-dependent tests. ([#1220](#1220)) by @jeff-hykin - **e2e test marker** — Marked end-to-end tests for selective CI runs. ([#1110](#1110)) by @paul-nechifor - **pytest stdin fix** — Added `-s` to default addopts for LCM autoconf compatibility. ([#1320](#1320)) by @spomichter --- ##⚠️ Breaking Changes - **RPC renames**: `ModuleBlueprint` → `_BlueprintAtom`, `ModuleBlueprintSet` → `Blueprint`, `ModuleConnection` → `Stream` ([#1143](#1143)) - **Image class rewrite**: `CudaImage` and `NumpyImage` removed. Image is now a pure NumPy dataclass. Methods like `solve_pnp`, `csrt_tracker`, `from_depth`, `to_depth_meters` removed. ([#1161](#1161)) - **ROS messages removed from core**: All `to_ros`/`from_ros` conversion methods removed. Use `ROSTransport` instead. ([#1230](#1230)) - **QuestButtons → Buttons**: Renamed with generic field names. ([#1261](#1261)) - **RerunBridge replaces old rerun init**: `dimos.dashboard.rerun_init` removed. Use `RerunBridgeModule` or the `rerun-bridge` CLI. ([#1154](#1154)) - **Unitree directory restructuring**: `unitree_go2` → `unitree/go2`, `unitree_g1` → `unitree/g1`. Blueprint names updated. ([#1221](#1221)) - **Default viewer is now rerun-web**: Use `--viewer-backend rerun` to restore native viewer. ([#1324](#1324)) --- ## Quickstart ```bash # Install uv pip install dimos[base,unitree] # Try it (no hardware needed) # NOTE: First run downloads ~2.4 GB from LFS dimos --replay run unitree-go2 # Simulate uv pip install dimos[base,unitree,sim] dimos --simulation run unitree-go2 ``` --- ## New Contributors 🎉 - @ruthwikdasyam — Quest VR teleoperation, phone teleop, arm teleop, webcam rerun viz - @JalajShuklaSS — GraspGen integration - @jca0 — MuJoCo simulation module - @christiefhyang — PR template --- **Full Changelog**: [v0.0.9...v0.0.10](v0.0.9...v0.0.10)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
LOCALIZATION_METHOD)MAP_PATHsupport for localization with pre-built PCD mapsros-joyruntime dependencyTest plan
MAP_PATHverifieduse_sim_timeverifiedLinear Issue
closes DIM-459