fix: add missing viewport.window + hydra_texture deps for headless rendering

[pv-nvidia]

Description

The headless rendering experience file (isaaclab.python.headless.rendering.kit) is missing two extensions required for the annotator/render-product pipeline to function in environments without a display (DGX Cloud, CI containers, SSH sessions):

• omni.kit.viewport.window: provides viewport infrastructure needed by rep.create.render_product_tiled() to create render products headlessly. Without it, rep.orchestrator.step() triggers an infinite createViewport loop, and annotators produce empty buffers.
• omni.kit.hydra_texture: provides the HydraTexture backend that replicator annotators rely on to extract rendered pixel data. While transitively pulled via omni.replicator.core → omni.syntheticdata, the load order in headless mode causes omni.hydratexture to be imported before the extension has started, making ht_available = False permanently.

The non-headless experience file (isaaclab.python.kit) already includes omni.kit.viewport.window. Adding both as explicit dependencies ensures correct startup ordering.

Type of change

• Bug fix (non-breaking change that fixes an issue)

Reproduction

Any Camera/TiledCamera test that actually renders in headless mode fails:

# On a headless machine (no DISPLAY, no nvidia_drm):
DISPLAY= python -m pytest source/isaaclab/test/sensors/test_tiled_camera.py -v
# test_tiled_camera_basic_functionality[cuda:0] → FAILED (cudaErrorIllegalAddress)

Fix

 # Rendering
 "omni.kit.material.library" = {}
 "omni.kit.viewport.rtx" = {}
+# Headless viewport + texture support — required for annotators/render-products
+# without a visible window (e.g. DGX Cloud, CI containers).
+"omni.kit.viewport.window" = {}
+"omni.kit.hydra_texture" = {}

Testing

After the fix, all 8 tiled camera test variants pass:

test_tiled_camera_deprecation_warning[cuda:0]    PASSED
test_tiled_camera_deprecation_warning[cpu]        PASSED
test_tiled_camera_cfg_deprecation_warning[cuda:0] PASSED
test_tiled_camera_cfg_deprecation_warning[cpu]    PASSED
test_tiled_camera_is_camera_subclass[cuda:0]      PASSED
test_tiled_camera_is_camera_subclass[cpu]         PASSED
test_tiled_camera_basic_functionality[cuda:0]     PASSED
test_tiled_camera_basic_functionality[cpu]        PASSED

Tested on DGX Cloud (2x L40, Kit 110.1.1, Isaac Sim v6.0.0-alpha.183, Vulkan headless).

[isaaclab-review-bot]

🤖 Isaac Lab Review Bot

Summary

This PR adds two missing Omniverse Kit extension dependencies (omni.kit.viewport.window and omni.kit.hydra_texture) to the headless rendering experience file to fix a startup ordering issue that causes camera annotators to produce empty buffers in truly headless environments (no display). The fix is minimal, targeted, and addresses a real initialization race condition in the Kit extension loading system.

Architecture Impact

Self-contained. This change only affects the .kit experience file used for headless rendering scenarios. It does not modify any Python code, APIs, or runtime behavior — it solely ensures correct extension load ordering at startup. The non-headless experience file (isaaclab.python.kit) already includes omni.kit.viewport.window, so this brings the headless variant to parity. No downstream code changes are required.

Implementation Verdict

Ship it. The fix is correct, minimal, and well-documented in both the PR description and inline comments.

Test Coverage

The PR description demonstrates thorough manual testing:

• Before: test_tiled_camera_basic_functionality[cuda:0] fails with cudaErrorIllegalAddress
• After: All 8 tiled camera test variants pass

However, there is no automated CI enforcement that these tests run in a truly headless environment (DISPLAY unset, no nvidia_drm). The existing test suite likely runs with a display available, which masks this class of bugs.

🟡 Suggestion: Consider adding a CI job that explicitly unsets DISPLAY and runs camera tests to catch regressions in headless rendering support. This is not blocking for this PR, but would prevent future regressions.

CI Status

No CI checks available yet. Once CI runs, verify that camera/tiled camera tests pass — though note that standard CI environments may have display access, which wouldn't exercise the exact failure mode this PR fixes.

Findings

🔵 Improvement: apps/isaaclab.python.headless.rendering.kit:25-26 — Consider explicit ordering if load order is critical

The PR description mentions that "the load order in headless mode causes omni.hydratexture to be imported before the extension has started." While adding explicit dependencies typically ensures correct ordering in Kit's dependency resolver, if there's a specific ordering requirement between omni.kit.viewport.window and omni.kit.hydra_texture, consider documenting which must load first, or verify Kit's resolver handles this correctly. The current placement (viewport.window before hydra_texture) appears intentional and likely correct.

🔵 Improvement: Documentation — Consider adding troubleshooting note

For future maintainers, it would be valuable to document in the Isaac Lab troubleshooting guide that headless rendering requires this specific experience file and what symptoms indicate missing extensions (infinite createViewport loop, empty annotator buffers). This is not blocking.

---

Final assessment: Clean, minimal fix for a real bug. The inline comments adequately explain why these dependencies are needed. The change aligns with the existing pattern in the non-headless experience file. No concerns with merging.

[greptile-apps]

Greptile Summary

This PR adds two missing extension dependencies (omni.kit.viewport.window and omni.kit.hydra_texture) to the headless rendering experience file (isaaclab.python.headless.rendering.kit) to fix annotator/render-product failures in display-less environments. The non-headless counterpart (isaaclab.python.kit) already includes omni.kit.viewport.window at line 82, and the fix correctly mirrors that pattern for the headless path.

Confidence Score: 5/5

Safe to merge — minimal, targeted config change that fixes a confirmed headless rendering regression with no P0/P1 findings.

The change adds exactly two extension entries to a single .kit config file, directly mirrors the pattern already used in the non-headless counterpart, is accompanied by clear inline comments, and is backed by a full 8/8 test pass on the target hardware. No logic, no runtime code, no schema changes.

No files require special attention.

Important Files Changed

Filename	Overview
apps/isaaclab.python.headless.rendering.kit	Adds omni.kit.viewport.window and omni.kit.hydra_texture to fix headless annotator/render-product pipeline; change is minimal, well-commented, and mirrors the non-headless kit's existing dependency.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["isaaclab.python.headless.rendering.kit"] --> B["isaaclab.python.headless (dep)"]
    A --> C["omni.replicator.core (dep)"]
    A --> D["omni.kit.material.library (dep)"]
    A --> E["omni.kit.viewport.rtx (dep)"]
    A --> F["omni.kit.viewport.window ✅ NEW"]
    A --> G["omni.kit.hydra_texture ✅ NEW"]

    C --> H["omni.syntheticdata (transitive)"]
    H --> I["omni.hydratexture module\n(Python import)"]

    G -->|"ensures extension\nstarted before import"| I

    F -->|"provides viewport infra\nfor render products"| J["rep.create.render_product_tiled()"]
    J --> K["rep.orchestrator.step()"]

    style F fill:#2d6a2d,color:#fff
    style G fill:#2d6a2d,color:#fff

Loading

_{Reviews (1): Last reviewed commit: "fix: add missing viewport.window + hydra..." | Re-trigger Greptile}

[pbarejko]

omni.ki.viewport.window brings omni.ui this is too heavy. I will make a PR for this.