Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion source/isaaclab/config/extension.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[package]

# Note: Semantic Versioning is used: https://semver.org/
version = "4.5.16"
version = "4.5.17"

# Description
title = "Isaac Lab framework for Robot Learning"
Expand Down
11 changes: 11 additions & 0 deletions source/isaaclab/docs/CHANGELOG.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,17 @@
Changelog
---------

4.5.17 (2026-03-18)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Changelog format looks correct — follows the existing pattern (version, date, section header, bullet). Minor note: the date 2026-03-18 is in the past relative to the PR creation — just verify this is intentional per your release process.

~~~~~~~~~~~~~~~~~~~

Fixed
^^^^^

* Fixed :meth:`~isaaclab.sim.SimulationContext.render` not calling ``app.update()`` when
running with Isaac Sim (Kit) and no active visualizer pumps the Kit app loop. This caused
``--video`` recording to produce black frames when not using ``--viz kit``.


4.5.16 (2026-03-10)
~~~~~~~~~~~~~~~~~~~

Expand Down
15 changes: 15 additions & 0 deletions source/isaaclab/isaaclab/sim/simulation_context.py
Original file line number Diff line number Diff line change
Expand Up @@ -604,6 +604,21 @@ def render(self, mode: int | None = None) -> None:
for callback in self._render_callbacks.values():
callback(None) # Pass None as event data

# When running with Isaac Sim (Kit) and no active visualizer already pumps the Kit
# app loop, call app.update() so the viewport and replicator render products
# (used e.g. by gym.wrappers.RecordVideo with render_mode="rgb_array") are refreshed.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Performance concern: unconditional app.update() on every render

app.update() is expensive — it pumps the entire Kit event loop (USD hydra, viewport compositing, extension ticks, etc.). This now runs on every render() call, even when:

  1. No video recording is active (RecordVideo wrapper not used)
  2. No replicator render products exist
  3. The user is just doing headless RL training with --video flag absent

Consider gating this more tightly. At minimum, checking self._has_offscreen_render or self.get_setting('/isaaclab/render/rtx_sensors') would limit the call to cases where render products actually exist. The has_kit() check alone is too broad — Kit is always present in Isaac Sim mode, even for pure headless training.

Alternatively, a dedicated "needs_app_pump" flag set when RecordVideo or camera sensors are initialized would be more precise and avoid per-frame setting lookups.

# KitVisualizer.pumps_app_update() returns True and calls app.update() in its own
# step(), so we skip this call to avoid double-rendering in that case.
if has_kit() and not any(v.pumps_app_update() for v in self._visualizers):
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Nit: redundant import + get_app() when has_kit() already confirmed Kit is running

has_kit() (in version.py) already calls sys.modules.get('omni.kit.app')mod.get_app() and confirms the result is not None. Re-importing and re-calling get_app() here is redundant. You could simplify:

import sys
mod = sys.modules['omni.kit.app']  # guaranteed present by has_kit()
app = mod.get_app()                 # guaranteed non-None by has_kit()
# app.is_running() is still worth checking
if app.is_running():
    app.update()

Not blocking, but it would make the code cleaner and avoid the misleading except ImportError.

try:
import omni.kit.app

app = omni.kit.app.get_app()
if app is not None and app.is_running():
app.update()
except (ImportError, AttributeError):
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Critical: Missing playSimulations=False guard — potential double physics step

Calling app.update() while the timeline is playing and /app/player/playSimulations is True (the default) will cause Kit to pump its internal physics loop, resulting in a double physics step per frame. This is exactly what KitVisualizer.step() guards against:

# KitVisualizer.step() does this correctly:
settings.set_bool('/app/player/playSimulations', False)
app.update()
settings.set_bool('/app/player/playSimulations', True)

The fix here should mirror that pattern:

if has_kit() and not any(v.pumps_app_update() for v in self._visualizers):
    try:
        import omni.kit.app
        app = omni.kit.app.get_app()
        if app is not None and app.is_running():
            self.set_setting('/app/player/playSimulations', False)
            app.update()
            self.set_setting('/app/player/playSimulations', True)
    except (ImportError, AttributeError):
        pass

Without this, every env.step()sim.step()render() cycle runs physics twice, causing 2× simulation speed, energy non-conservation, and incorrect RL rewards.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes we should have the playSimulations guard. could this also be moved to the kit visualizer script so that we don't have kit-specific logic in the simulation context?

pass

def update_visualizers(self, dt: float) -> None:
"""Update visualizers without triggering renderer/GUI."""
if not self._visualizers:
Expand Down
61 changes: 61 additions & 0 deletions source/isaaclab/test/sim/test_simulation_context.py
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,67 @@ def test_render():
assert sim.is_playing()


@pytest.mark.isaacsim_ci
def test_render_pumps_app_update_without_visualizer():
"""Regression test for issue #5052: render() must call app.update() when no visualizer pumps the Kit loop.

Without this call, replicator render products (used by gym.wrappers.RecordVideo for
rgb_array rendering) are never updated, producing black video frames.
"""
from unittest.mock import MagicMock, patch

cfg = SimulationCfg(dt=0.01)
sim = SimulationContext(cfg)
sim.reset()

mock_app = MagicMock()
mock_app.is_running.return_value = True

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Test gap: mock patches omni.kit.app.get_app but has_kit() is not mocked

The has_kit() function checks sys.modules.get('omni.kit.app') and calls the real get_app() — it doesn't use the patched version from unittest.mock.patch('omni.kit.app.get_app', ...). In CI where Kit IS running, has_kit() returns True from the real module, so this test passes. But in a kitless test environment, has_kit() would return False and the patched get_app would never be reached, making the assertion vacuously pass.

Consider also patching has_kit to make the test robust in all environments:

with patch('isaaclab.sim.simulation_context.has_kit', return_value=True), \
     patch('omni.kit.app.get_app', return_value=mock_app):
    sim.render()

Since these tests are gated by @pytest.mark.isaacsim_ci, this is not a blocker — but it's worth noting for maintainability.

with patch("omni.kit.app.get_app", return_value=mock_app):
sim.render()

# app.update() must be called when no visualizer pumps the Kit app loop
mock_app.update.assert_called_once()


@pytest.mark.isaacsim_ci
def test_render_skips_app_update_when_visualizer_pumps_it():
"""Regression test: render() must NOT call app.update() when a visualizer already does.

A visualizer that returns ``pumps_app_update() == True`` (e.g. KitVisualizer) calls
``app.update()`` in its own ``step()``, so ``SimulationContext.render()`` must not
call it again to avoid double-rendering.
"""
from unittest.mock import MagicMock, patch

from isaaclab.visualizers.base_visualizer import BaseVisualizer

cfg = SimulationCfg(dt=0.01)
sim = SimulationContext(cfg)
sim.reset()

# Inject a mock visualizer that pumps the app update
mock_viz = MagicMock(spec=BaseVisualizer)
mock_viz.pumps_app_update.return_value = True
mock_viz.is_closed = False
mock_viz.is_running.return_value = True
mock_viz.is_rendering_paused.return_value = False
mock_viz.is_training_paused.return_value = False
mock_viz.get_rendering_dt.return_value = None
sim._visualizers = [mock_viz]
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Test improvement: verify update_visualizers interaction with mock visualizer

This test injects a mock visualizer into _visualizers but doesn't fully account for what update_visualizers() does with it. update_visualizers() calls viz.is_closed, viz.is_running(), viz.is_rendering_paused(), viz.is_training_paused(), and viz.step(dt) — all of which are mocked. However, update_scene_data_provider() is also called, which accesses _scene_data_provider. If a future refactor changes update_visualizers' behavior, this test could break in non-obvious ways.

Consider also asserting that mock_viz.step.assert_called() to verify the visualizer's step() method (which is where KitVisualizer does its own app.update()) was actually invoked, confirming the intended code path.


mock_app = MagicMock()
mock_app.is_running.return_value = True

with patch("omni.kit.app.get_app", return_value=mock_app):
sim.render()

# app.update() must NOT be called since the visualizer already pumps it
mock_app.update.assert_not_called()

sim._visualizers = []

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Good: Cleaning up sim._visualizers = [] after the test to avoid leaking mock state into the teardown's clear_instance(). Defensive and correct.


"""
Stage Operations Tests
"""
Expand Down