Questions Regarding Residual RL Fine-Tuning Process and Checkpoints

[winter12279]

Hi Compass Team,

Thank you for the great work and your previous support!

We are currently planning to fine-tune the model using residual RL with our environments and embodiments, as recommended in the GitHub repository:

The generalist policy uses one-hot embodiment encoding and may not generalize perfectly to unseen embodiment types. For best results with new embodiment types, we recommend fine-tuning with residual RL first.

To move forward, we’d like to clarify a few points:

1. Could you guys provide detailed instructions or documentation on how to perform the residual RL fine-tuning process?
2. What .ckpt files or related resources are required for fine-tuning? Would be great if you could point us to the right ones.

We’d really appreciate any guidance or materials you could share to help us get started.

Thanks again!

[nv-liuw]

Hi @winter12279,

Thanks for your interest.

To use COMPASS for fine tuning, you should first follow the instruction here https://github.com/NVlabs/COMPASS?tab=readme-ov-file#add-new-embodiment-or-scene to bring up your own embodiment and environment digital twin.

Then, you can follow https://github.com/NVlabs/COMPASS?tab=readme-ov-file#residual-rl-specialists to train and eval the specialist for your own embodiment.

After you the specialist policy ready, you can deploy it with ONNX conversion by following: https://github.com/NVlabs/COMPASS?tab=readme-ov-file#model-export

The generalist policy ckpt is only a example for user to try out, we don't need it for fine tuning. For fine tuning, we need the X-Mobility ckpt (https://github.com/NVlabs/COMPASS?tab=readme-ov-file#5-x-mobility-installation) as the base policy to optimize.