GraspGen generator with docker dimos module integration

[JalajShuklaSS]

Adds GraspGen grasp generation to DimOS and introduces Docker-backed module execution, enabling dependency-heavy modules to run in containers while behaving like native modules with RPC, streaming, and autoconnect support. This skill would further be a subskill inside the manipulation stack to generate grasps.

GraspGen Module

• Adds a GraspGen module for grasp generation.
• Generates grasp candidates from pointcloud inputs and publishes grasp poses via streams
• Exposes lifecycle control and grasp generation via RPC
• Designed to run inside Docker to isolate CUDA, PyTorch, and EGL dependencies avoiding ABI conflicts
• Having a temporary test demo, composable with existing perception and manipulation pipelines

Docker-backed Module Support

• Introduces DockerModule (host side handler) and StandaloneModuleRunner (container runtime)
• Enables containerized modules to behave like native modules from the blueprint perspective
• Supports RPC communication via LCM multicast
• Enables stream wiring via configure_stream, compatible with existing autoconnect logic
• Current stream support uses pLCMTransport (topic-based pub/sub)
• Once this spec and schematic is discussed can add full transport parity (SHM, JPEG-SHM, typed LCM)

[greptile-apps]

Greptile Overview

Greptile Summary

This PR introduces Docker-backed module execution to DimOS and adds GraspGen grasp generation capability. The implementation enables dependency-heavy modules (CUDA, PyTorch, EGL) to run in isolated containers while behaving like native modules from the blueprint perspective.

Key Changes:

• Docker Module Infrastructure: New DockerModule host-side handler and StandaloneModuleRunner for containerized execution with RPC communication via LCM multicast
• Dockerfile Conversion: Automated footer injection system (docker_build.py, module-install.sh) that converts any Dockerfile into a DimOS module container
• GraspGen Integration: Adds GraspGenModule with grasp pose generation from pointclouds, supporting multiple grippers (Robotiq 2F-140, Franka Panda, suction cup)
• Perception Extensions: Enhanced ObjectSceneRegistrationModule with pointcloud extraction methods for grasp generation support
• Stream Wiring: New configure_stream RPC method enables Docker modules to participate in autoconnect stream wiring
• End-to-End Demo: Complete demo_perception_grasping blueprint integrating camera → perception → grasp generation → agent

Issues Found:

• Path mismatch between Dockerfile (GRASPGEN_PATH=/app/GraspGen) and Python default (/dimos/third_party/GraspGen) will cause initialization failures
• Config file search returns non-existent path instead of raising error
• Personal test file objscene_registreation_myversion.py with typo should not be committed
• Temporary testing module temp_graspgen_testing.py marked as temporary but being merged
• PoseArray uses insecure pickle encoding instead of proper LCM schema

Architecture Notes:
The Docker module deployment bypasses standard Dask actor deployment, which may affect existing lifecycle hooks. The implementation uses host networking for LCM multicast, requiring --network=host mode.

Confidence Score: 3/5

• This PR has solid architecture but contains critical path mismatch bug and includes temporary/test files that should not be merged
• Score of 3 reflects well-designed Docker module infrastructure and comprehensive implementation, but the GRASPGEN_PATH mismatch between Dockerfile and Python code will cause runtime failures. Additionally, the inclusion of personal test files and temporary modules reduces production readiness.
• Pay close attention to dimos/grasping/graspgen_module.py (path mismatch bug), dimos/perception/objscene_registreation_myversion.py (should be removed), and dimos/grasping/temp_graspgen_testing.py (temporary code)

Important Files Changed

Filename	Overview
dimos/core/docker_module.py	Adds DockerModule host-side handler and StandaloneModuleRunner for containerized module execution with RPC communication
dimos/grasping/graspgen_module.py	Implements GraspGenModule with Docker-based execution for grasp pose generation with CUDA dependencies
dimos/core/init.py	Integrates Docker module deployment into DimosCluster with automatic detection and lifecycle management
dimos/grasping/temp_graspgen_testing.py	Temporary test pipeline wiring perception to grasp generation via RPC calls (marked as temporary)
dimos/msgs/geometry_msgs/PoseArray.py	Adds PoseArray message type with pickle-based encoding/decoding for grasp pose arrays
dimos/perception/objscene_registreation_myversion.py	Appears to be a personal copy of object_scene_registration with typo in filename (should not be committed)

Sequence Diagram

sequenceDiagram
    participant User
    participant Agent
    participant GraspingSkill
    participant GraspPipeline
    participant Perception
    participant GraspGen
    participant Docker
    
    Note over User,Docker: Initialization Phase
    User->>Agent: Start demo_perception_grasping blueprint
    Agent->>Docker: Deploy GraspGenModule
    Docker->>Docker: Build image with DimOS footer
    Docker->>GraspGen: Start container with LCMRPC
    GraspGen-->>Docker: Module ready signal
    Docker-->>Agent: DockerModule handle created
    Agent->>Perception: Deploy ObjectSceneRegistration
    Agent->>GraspPipeline: Deploy GraspPipeline
    Agent->>GraspingSkill: Deploy GraspingSkillContainer
    
    Note over User,Docker: Grasp Generation Flow
    User->>Agent: "Generate grasps for cup"
    Agent->>GraspingSkill: generate_grasps("cup")
    GraspingSkill->>GraspPipeline: Publish trigger ("cup")
    GraspPipeline->>Perception: RPC: get_object_pointcloud_by_name("cup")
    Perception-->>GraspPipeline: PointCloud2
    GraspPipeline->>Perception: RPC: get_full_scene_pointcloud()
    Perception-->>GraspPipeline: Scene PointCloud2
    GraspPipeline->>GraspGen: RPC: generate_grasps(pc, scene_pc)
    GraspGen->>GraspGen: Initialize GraspGen model (if needed)
    GraspGen->>GraspGen: Run inference with CUDA
    GraspGen->>GraspGen: Filter collisions (optional)
    GraspGen->>GraspingSkill: Publish PoseArray via stream
    GraspingSkill->>GraspingSkill: _on_grasps() callback
    GraspingSkill-->>Agent: Return grasp summary
    Agent-->>User: "Generated 100 grasps for 'cup'"
    
    Note over User,Docker: Cleanup Phase
    User->>Agent: Stop blueprint
    Agent->>Docker: Stop all DockerModules
    Docker->>GraspGen: Send SIGTERM
    GraspGen->>GraspGen: Cleanup GPU memory
    Docker->>Docker: Remove container

Loading

[greptile-apps]

_{10 files reviewed, 10 comments}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

Additional Comments (1)

dimos/perception/object_scene_registration.py
The parameter is exclude_object_id but usage in the grasp pipeline passes object names. Verify the parameter semantics match the actual usage.

[spomichter]

We need to move away from using skill containers. Skills should just be in the Module itself.

[mustafab0]

Please delete temp files before merging

[paul-nechifor]

@JalajShuklaSS You have a few mypy failures and test failures. I can help with mypy if you need me to.