Adds time-based mdp (observation) functions by TheIndoorDad · Pull Request #2332 · isaac-sim/IsaacLab

TheIndoorDad · 2025-04-17T20:09:59Z

Description

As discussed in #2284, I was writing an observation function to pass the time remaining in an episode (in seconds) to an observation term in a Manager-based environment, and found this could not be done without modifying ManagerBasedRLEnv to initialize episode_length_buf before managers are loaded. Here is a summary of changes made:

Added initialization of episode_length_buf in :meth:load_managers() of :class:~isaaclab.envs.ManagerBasedRLEnv to make it available for use in mdp functions. Note: existing initialization in :meth:__init__ left in place in case it is needed for other use cases. Potentially redundant? Assess.
Added :attr:~isaaclab.envs.ManagerBasedRLEnv.curr_episode_length to :class:~isaaclab.envs.ManagerBasedRLEnv which returns reshaped episode_length_buf so it is visible as an attribute in the documentation.
Added time observation functions to ~isaaclab.envs.mdp.observations module, :func:~isaaclab.envs.mdp.observations.current_time_s and :func:~isaaclab.envs.mdp.observations.remaining_time_s.

I'm not certain whether the documentation will be updated automatically or if there are further steps I need to take. When I build the documentation on my machine it is updated, but the outputs are ignored by git. Please let me know if there's anything else I need to do.

I could also use some advice on tests (apologies in advance for my lack of experience here, my background is not in software dev). Locally I modified the Isaac-Velocity-Rough-Anymal-C-v0 task to add the two new observation functions, and began to train a policy in rsl_rl using the provided scripts/reinforcement_learning/rsl_rl/train.py script, and both were available to be viewed and appeared to be working correctly. I tried to run the existing suite of unit tests but it gave me an error I don't understand (see below). I also started to create a new script similar to isaaclab/test/envs/check_manager_based_env_anymal_locomotion.py but that would have required a policy trained using the new observation functions (which I can produce, but wasn't sure if that would be worthwhile since it wouldn't be available to others).

Output when running ./isaaclab.sh -t:

[INFO] Warm starting the simulation app before running tests.                                                                                                                                                                                                                                                                        
ERROR:root:Error warm starting the app: b'2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 16 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 17 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 18 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 19 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 20 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 21 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 22 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 23 belongs to.\nMESA-INTEL: warning: Performance support disabled, consider sysctl dev.i915.perf_stream_paranoid=0\n\n'

Cheers.

Type of change

New feature (non-breaking change which adds functionality)
This change requires a documentation update

Checklist

I have run the pre-commit checks with ./isaaclab.sh --format
I have made corresponding changes to the documentation (maybe?)
My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works (please advise)
I have updated the changelog and the corresponding version in the extension's config/extension.toml file
I have added my name to the CONTRIBUTORS.md or my name already exists there

…t available for use by mdp functions.

…h_buf so it is visible as an attribute in the documentation.

…f ManagerBasedRLEnv.

Mayankm96 · 2025-04-25T15:42:23Z


        # initialize data and constants
        # -- init buffers
+        # TODO: this may be redundant since self.episode_length_buf is now initialized in load_managers() to make it


Instead of moving it to load_managers (which is only for managers and not buffers), maybe it is better to move the buffer initialization further up in the class

I think I can move it above the super().__init__ call if I change the parameters to be self.cfg.scene.num_envs and self.cfg.sim.device. Seems to be working on my side. I'll make a commit for your review.

Mayankm96 · 2025-04-25T15:43:19Z

        return math.ceil(self.max_episode_length_s / self.step_dt)

+    @property
+    def curr_episode_length(self) -> torch.Tensor:


episode_length_buf is a variable you can access outside as it is not a private member. I don't think making a property for it is necessary at this point.

Sure. I did this purely so the property would show up as an attribute in the documentation, but if you don't think that's necessary (or there's a better way to do it) I will remove.

TheIndoorDad

Commits to follow with above suggested changes.

TheIndoorDad · 2025-04-25T17:52:40Z

        return math.ceil(self.max_episode_length_s / self.step_dt)

+    @property
+    def curr_episode_length(self) -> torch.Tensor:


Sure. I did this purely so the property would show up as an attribute in the documentation, but if you don't think that's necessary (or there's a better way to do it) I will remove.

TheIndoorDad · 2025-04-25T18:01:02Z


        # initialize data and constants
        # -- init buffers
+        # TODO: this may be redundant since self.episode_length_buf is now initialized in load_managers() to make it


I think I can move it above the super().__init__ call if I change the parameters to be self.cfg.scene.num_envs and self.cfg.sim.device. Seems to be working on my side. I'll make a commit for your review.

…f directly as suggested. Co-authored-by: Mayank Mittal <12863862+Mayankm96@users.noreply.github.com> Signed-off-by: TheIndoorDad <167908515+TheIndoorDad@users.noreply.github.com>

…re `super().__init__` call.

Signed-off-by: Mayank Mittal <12863862+Mayankm96@users.noreply.github.com>

TheIndoorDad · 2025-05-21T19:19:15Z

Hi @Mayankm96, is there anything else you need me to do to get this PR committed to the main branch? Sorry, I'm just not sure of the process.

Cheers, and thanks for your help.

Signed-off-by: Mayank Mittal <12863862+Mayankm96@users.noreply.github.com>

# Description As discussed in isaac-sim#2284, I was writing an observation function to pass the time remaining in an episode (in seconds) to an observation term in a Manager-based environment, and found this could not be done without modifying `ManagerBasedRLEnv` to initialize `episode_length_buf` before managers are loaded. Here is a summary of changes made: * Added initialization of `episode_length_buf` in :meth:`load_managers()` of :class:`~isaaclab.envs.ManagerBasedRLEnv` to make it available for use in mdp functions. Note: existing initialization in :meth:`__init__` left in place in case it is needed for other use cases. Potentially redundant? Assess. * Added :attr:`~isaaclab.envs.ManagerBasedRLEnv.curr_episode_length` to :class:`~isaaclab.envs.ManagerBasedRLEnv` which returns reshaped ``episode_length_buf`` so it is visible as an attribute in the documentation. * Added time observation functions to `~isaaclab.envs.mdp.observations` module, :func:`~isaaclab.envs.mdp.observations.current_time_s` and :func:`~isaaclab.envs.mdp.observations.remaining_time_s`. I'm not certain whether the documentation will be updated automatically or if there are further steps I need to take. When I build the documentation on my machine it is updated, but the outputs are ignored by git. Please let me know if there's anything else I need to do. I could also use some advice on tests (apologies in advance for my lack of experience here, my background is not in software dev). Locally I modified the `Isaac-Velocity-Rough-Anymal-C-v0` task to add the two new observation functions, and began to train a policy in rsl_rl using the provided `scripts/reinforcement_learning/rsl_rl/train.py` script, and both were available to be viewed and appeared to be working correctly. I tried to run the existing suite of unit tests but it gave me an error I don't understand (see below). I also started to create a new script similar to [`isaaclab/test/envs/check_manager_based_env_anymal_locomotion.py`](https://github.com/isaac-sim/IsaacLab/blob/7de6d6fef9424c95fc68dc767af67ffbe0da6832/source/isaaclab/test/envs/check_manager_based_env_anymal_locomotion.py) but that would have required a policy trained using the new observation functions (which I can produce, but wasn't sure if that would be worthwhile since it wouldn't be available to others). Output when running `./isaaclab.sh -t`: ``` [INFO] Warm starting the simulation app before running tests. ERROR:root:Error warm starting the app: b'2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 16 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 17 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 18 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 19 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 20 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 21 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 22 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 23 belongs to.\nMESA-INTEL: warning: Performance support disabled, consider sysctl dev.i915.perf_stream_paranoid=0\n\n' ``` Cheers. ## Type of change - New feature (non-breaking change which adds functionality) - This change requires a documentation update ## Checklist - [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with `./isaaclab.sh --format` - [x] I have made corresponding changes to the documentation (_maybe_?) - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works (please advise) - [x] I have updated the changelog and the corresponding version in the extension's `config/extension.toml` file - [x] I have added my name to the `CONTRIBUTORS.md` or my name already exists there --------- Signed-off-by: TheIndoorDad <167908515+TheIndoorDad@users.noreply.github.com> Signed-off-by: Mayank Mittal <12863862+Mayankm96@users.noreply.github.com> Co-authored-by: Mayank Mittal <12863862+Mayankm96@users.noreply.github.com>

# Description As discussed in isaac-sim#2284, I was writing an observation function to pass the time remaining in an episode (in seconds) to an observation term in a Manager-based environment, and found this could not be done without modifying `ManagerBasedRLEnv` to initialize `episode_length_buf` before managers are loaded. Here is a summary of changes made: * Added initialization of `episode_length_buf` in :meth:`load_managers()` of :class:`~isaaclab.envs.ManagerBasedRLEnv` to make it available for use in mdp functions. Note: existing initialization in :meth:`__init__` left in place in case it is needed for other use cases. Potentially redundant? Assess. * Added :attr:`~isaaclab.envs.ManagerBasedRLEnv.curr_episode_length` to :class:`~isaaclab.envs.ManagerBasedRLEnv` which returns reshaped ``episode_length_buf`` so it is visible as an attribute in the documentation. * Added time observation functions to `~isaaclab.envs.mdp.observations` module, :func:`~isaaclab.envs.mdp.observations.current_time_s` and :func:`~isaaclab.envs.mdp.observations.remaining_time_s`. I'm not certain whether the documentation will be updated automatically or if there are further steps I need to take. When I build the documentation on my machine it is updated, but the outputs are ignored by git. Please let me know if there's anything else I need to do. I could also use some advice on tests (apologies in advance for my lack of experience here, my background is not in software dev). Locally I modified the `Isaac-Velocity-Rough-Anymal-C-v0` task to add the two new observation functions, and began to train a policy in rsl_rl using the provided `scripts/reinforcement_learning/rsl_rl/train.py` script, and both were available to be viewed and appeared to be working correctly. I tried to run the existing suite of unit tests but it gave me an error I don't understand (see below). I also started to create a new script similar to [`isaaclab/test/envs/check_manager_based_env_anymal_locomotion.py`](https://github.com/isaac-sim/IsaacLab/blob/31f3729035b6e2a37797e84295f0a249352a3f82/source/isaaclab/test/envs/check_manager_based_env_anymal_locomotion.py) but that would have required a policy trained using the new observation functions (which I can produce, but wasn't sure if that would be worthwhile since it wouldn't be available to others). Output when running `./isaaclab.sh -t`: ``` [INFO] Warm starting the simulation app before running tests. ERROR:root:Error warm starting the app: b'2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 16 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 17 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 18 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 19 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 20 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 21 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 22 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 23 belongs to.\nMESA-INTEL: warning: Performance support disabled, consider sysctl dev.i915.perf_stream_paranoid=0\n\n' ``` Cheers. ## Type of change - New feature (non-breaking change which adds functionality) - This change requires a documentation update ## Checklist - [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with `./isaaclab.sh --format` - [x] I have made corresponding changes to the documentation (_maybe_?) - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works (please advise) - [x] I have updated the changelog and the corresponding version in the extension's `config/extension.toml` file - [x] I have added my name to the `CONTRIBUTORS.md` or my name already exists there --------- Signed-off-by: TheIndoorDad <167908515+TheIndoorDad@users.noreply.github.com> Signed-off-by: Mayank Mittal <12863862+Mayankm96@users.noreply.github.com> Co-authored-by: Mayank Mittal <12863862+Mayankm96@users.noreply.github.com>

TheIndoorDad added 4 commits April 17, 2025 13:12

Add initialization of episode_length_buf to load_managers() to make i…

ac624da

…t available for use by mdp functions.

Add curr_episode_length property which returns reshaped episode_lengt…

d91b8cb

…h_buf so it is visible as an attribute in the documentation.

Add time observation functions, using curr_episode_length attribute o…

fdc6d16

…f ManagerBasedRLEnv.

Updated for time observation function additions.

7e87dfe

TheIndoorDad requested review from Mayankm96, jsmith-bdai and kellyguo11 as code owners April 17, 2025 20:10

Ran formatting script previously missed after updating changelog.

b6fd45b

Mayankm96 reviewed Apr 25, 2025

View reviewed changes

Comment thread source/isaaclab/isaaclab/envs/mdp/observations.py Outdated

Mayankm96 reviewed Apr 25, 2025

View reviewed changes

Comment thread source/isaaclab/isaaclab/envs/mdp/observations.py Outdated

Mayankm96 reviewed Apr 25, 2025

View reviewed changes

Comment thread source/isaaclab/isaaclab/envs/manager_based_rl_env.py Outdated

TheIndoorDad commented Apr 25, 2025

View reviewed changes

TheIndoorDad and others added 3 commits April 25, 2025 12:06

Remove curr_episode_length property and instead use episode_length_bu…

fd98496

…f directly as suggested. Co-authored-by: Mayank Mittal <12863862+Mayankm96@users.noreply.github.com> Signed-off-by: TheIndoorDad <167908515+TheIndoorDad@users.noreply.github.com>

Move episode_length_buf init back out of load_managers() and befo…

6e82a6e

…re `super().__init__` call.

Fixed formatting.

70b8a58

TheIndoorDad requested a review from Mayankm96 April 25, 2025 18:25

Mayankm96 reviewed May 7, 2025

View reviewed changes

Comment thread source/isaaclab/isaaclab/envs/manager_based_rl_env.py

Mayankm96 reviewed May 7, 2025

View reviewed changes

Comment thread source/isaaclab/isaaclab/envs/manager_based_rl_env.py Outdated

Mayankm96 added 2 commits May 7, 2025 08:13

Apply suggestions from code review

c8e314f

Signed-off-by: Mayank Mittal <12863862+Mayankm96@users.noreply.github.com>

Merge branch 'main' into feature/SV/time_observation

97552c5

Signed-off-by: Mayank Mittal <12863862+Mayankm96@users.noreply.github.com>

Mayankm96 approved these changes May 7, 2025

View reviewed changes

Mayankm96 added bug Something isn't working enhancement New feature or request labels May 7, 2025

Mayankm96 changed the title ~~Adds time-based mdp (observation) functions.~~ Adds time-based mdp (observation) functions May 7, 2025

Merge branch 'main' into feature/SV/time_observation

2899a3b

Signed-off-by: Mayank Mittal <12863862+Mayankm96@users.noreply.github.com>

Merge branch 'main' into feature/SV/time_observation

a919d2e

Signed-off-by: Mayank Mittal <12863862+Mayankm96@users.noreply.github.com>

Mayankm96 reviewed Jun 2, 2025

View reviewed changes

Comment thread source/isaaclab/config/extension.toml

Apply suggestions from code review

78228b3

Signed-off-by: Mayank Mittal <12863862+Mayankm96@users.noreply.github.com>

Mayankm96 merged commit dca126a into isaac-sim:main Jun 2, 2025
3 of 4 checks passed

hujc7 mentioned this pull request Apr 22, 2026

Supports Anymal-D Rough terrain training in newton #5225

Closed

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds time-based mdp (observation) functions#2332

Adds time-based mdp (observation) functions#2332
Mayankm96 merged 13 commits into
isaac-sim:mainfrom
TheIndoorDad:feature/SV/time_observation

TheIndoorDad commented Apr 17, 2025

Uh oh!

Mayankm96 Apr 25, 2025

Uh oh!

TheIndoorDad Apr 25, 2025

Uh oh!

Mayankm96 Apr 25, 2025

Uh oh!

TheIndoorDad Apr 25, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

TheIndoorDad left a comment

Uh oh!

TheIndoorDad Apr 25, 2025

Uh oh!

TheIndoorDad Apr 25, 2025

Uh oh!

Uh oh!

Uh oh!

TheIndoorDad commented May 21, 2025

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

TheIndoorDad commented Apr 17, 2025

Description

Type of change

Checklist

Uh oh!

Mayankm96 Apr 25, 2025

Choose a reason for hiding this comment

Uh oh!

TheIndoorDad Apr 25, 2025

Choose a reason for hiding this comment

Uh oh!

Mayankm96 Apr 25, 2025

Choose a reason for hiding this comment

Uh oh!

TheIndoorDad Apr 25, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

TheIndoorDad left a comment

Choose a reason for hiding this comment

Uh oh!

TheIndoorDad Apr 25, 2025

Choose a reason for hiding this comment

Uh oh!

TheIndoorDad Apr 25, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

TheIndoorDad commented May 21, 2025

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants