Skip to content

Test run time improvement: remove AppLauncher from tests that do not require it#5646

Merged
pbarejko merged 4 commits into
isaac-sim:developfrom
pbarejko:pbarejko/app-cleanup
May 16, 2026
Merged

Test run time improvement: remove AppLauncher from tests that do not require it#5646
pbarejko merged 4 commits into
isaac-sim:developfrom
pbarejko:pbarejko/app-cleanup

Conversation

@pbarejko
Copy link
Copy Markdown
Collaborator

Description

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context.
List any dependencies that are required for this change.

Fixes # (issue)

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (existing functionality will not work without user modification)
  • Documentation update

Screenshots

Please attach before and after screenshots of the change if applicable.

Checklist

  • I have read and understood the contribution guidelines
  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the changelog and the corresponding version in the extension's config/extension.toml file
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

@github-actions github-actions Bot added the isaac-lab Related to Isaac Lab team label May 15, 2026
Copy link
Copy Markdown

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Overview

This PR removes AppLauncher initialization code from 26 test files across multiple test modules (actuators, controllers, devices, envs, sensors, terrains, utils), enabling these tests to run without requiring the full Isaac Sim/Kit environment.

Key Findings

⚠️ Areas Requiring Attention

1. PR Description Incomplete
The PR description uses the default template without explaining:

  • The motivation behind this change
  • Which issue this fixes (if any)
  • Whether all affected tests have been verified to pass without AppLauncher

2. Tests with Potential Runtime Dependencies
Several of the modified tests import modules that may require the simulation context:

File Concern
test_configclass.py Original comment mentioned warp dependency requiring running simulation
test_dict.py Same warp dependency note in original code
test_string.py Same warp dependency note in original code
test_multi_mesh_ray_caster.py Original comment: "Used for warp"
test_mock_sensors.py Imports and uses warp as wp
test_math.py Original comment: "This is only needed because of warp dependency"

If these tests now run without AppLauncher, please confirm warp initialization is handled elsewhere (e.g., pytest fixtures, conftest.py) or that these specific tests no longer require warp.

3. Empty Changelog Skip Files
Three .skip files are added with only a newline. While this may be intentional to bypass changelog requirements, consider adding a comment explaining why no changelog entry is needed.

✅ Positive Aspects

  • Consistent Pattern Removal: The refactoring is systematic and touches only the AppLauncher boilerplate
  • No Logic Changes: Test logic remains unchanged, reducing regression risk
  • Faster Test Execution: Tests will run significantly faster without simulator initialization overhead

Checklist Items

Based on the PR checklist, these items appear incomplete:

  • Description explains the change and motivation
  • Pre-commit checks run
  • Tests verified to pass

Recommendations

  1. Update PR description with the rationale for this change
  2. Confirm test execution - share CI results showing all modified tests pass
  3. Document warp handling - clarify how tests using warp will initialize it
  4. Consider adding a changelog entry for this test infrastructure improvement

Verdict

This appears to be a work-in-progress for "kitless" test suites. The approach is sound, but the PR needs documentation and verification before merge. Would recommend addressing the above items and marking the checklist appropriately.


🔄 Update (e6cdd7d)

New commits add significant additional changes beyond the original kitless test suite scope:

  1. Unified RL training/play entrypoints - New isaaclab train / isaaclab play CLI commands with library dispatch (rsl_rl, sb3, skrl, etc.)
  2. AppLauncher CUDA fix - Deferred torch.cuda.set_device() to after SimulationApp starts to prevent OpenBLAS fork crash
  3. Converter refactoring - URDF/MJCF converters now delegate to Isaac Sim importers; removed urdf_utils.py
  4. RSL-RL export test batching - Batch Kit launches to reduce CI churn
  5. Newton label renaming fix - Boundary-safe path matching for label arrays
  6. Teleop XCR replay automation - New isaaclab_teleop.automation subpackage for CI replay
  7. Many changelog entries - Version bumps and documented fixes

⚠️ Scope creep concern: This PR now touches training scripts, CLI, converters, multiple extensions, and teleop — far beyond "speed test run". Consider splitting into focused PRs for easier review.

Previous review concerns about kitless tests still apply.


🔄 Update (665eb34)

⚠️ Major scope expansion: This commit adds 300 files touching many core subsystems. Key changes:

1. Camera/Sensor Warp Migration (High Impact)

  • CameraData now uses ProxyArray (warp+torch dual views) instead of raw torch tensors
  • Added create_buffers() to allocate pose/intrinsics as warp arrays
  • New warp kernels in warp_math.py for camera orientation conversion
  • Tests updated to access .torch on ProxyArray properties

2. Renderer Interface Changes (Breaking)

  • BaseRenderer.set_outputs() and update_camera() now accept ProxyArray instead of torch.Tensor
  • RenderBufferSpec dtype changed from torch.dtype to warp dtypes (wp.uint8, wp.float32, etc.)
  • Newton and OVRTX renderers updated to match

3. Import Refactoring (Breaking)

  • Changed ~50+ files from from isaaclab.utils import configclass to from isaaclab.utils.configclass import configclass
  • Affects sensors, spawners, terrains, envs, controllers, managers, etc.

4. More Kitless Tests

  • Additional test files modified to remove AppLauncher requirement
  • Added new tests: test_stacked_image_mdp.py, test_stacked_image_obs_manager.py

5. Newton Updates

  • Bumped newton pin from v1.2.0rc2 to v1.2.0 (stable)
  • Added IAdapter version verification in cubric shim
  • NewtonWarpRenderer updated for ProxyArray interface

6. OVRTX Renderer Improvements

  • Avoided disk I/O by using in-memory USD export
  • Renamed use_cloning to use_ovrtx_cloning, default True
  • Fixed multi-GPU device handling
  • Fixed scene partition attribute creation

7. Articulation Additions (isaaclab_ovphysx)

  • Major new Articulation and ArticulationData implementation
  • Dual *_index / *_mask write/set methods

8. Other Notable Changes

  • ProxyArray.__getattr__ now forwards unknown attributes to torch view
  • Pink IK test tolerance increased, rotation error formula fixed
  • Kit startup performance test adjusted for CI environment

🛑 Recommendations:

  1. Split this PR - 300 files across core sensors, renderers, tests, newton, ovrtx, and ovphysx is too much for one review
  2. Add migration guide - The configclass import change and ProxyArray changes are breaking
  3. CI verification needed - These camera/renderer changes need thorough testing

Previous concerns about PR scope and documentation still apply.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 15, 2026

Greptile Summary

This PR removes the Isaac Sim AppLauncher bootstrapping block from 26 test files that test pure-Python/PyTorch/Warp utilities and don't require a running simulation, eliminating the heavyweight Omniverse Kit startup overhead from those test suites. Three new .skip changelog stub files are added for isaaclab_contrib, isaaclab_rl, and isaaclab_teleop.

  • Kit-less test acceleration: All changed tests (actuators, controllers, devices, envs, sensors, terrains, utils) previously launched a headless Isaac Sim instance unconditionally; removing the launcher lets them run as ordinary pytest suites.
  • Warp-using tests now rely on lazy init: Three files (test_wrench_composer.py, test_mock_sensors.py, test_multi_mesh_ray_caster.py) still import and call warp APIs but omit wp.init(), unlike the established pattern in existing kit-less warp tests (test_proxy_array.py, test_ray_caster_kernels.py).

Confidence Score: 4/5

Safe to merge; changes are additive removals of boilerplate with no logic alterations to any tested code path.

The bulk of the changes are straightforward and correct — tests that only exercise pure-Python/torch utilities no longer spin up a simulation engine. The three files that still invoke warp APIs (test_wrench_composer.py, test_mock_sensors.py, test_multi_mesh_ray_caster.py) depend on warp's lazy initialization instead of the explicit wp.init() call used by other kit-less warp tests already in this repo. This is a minor consistency gap rather than a functional breakage.

test_wrench_composer.py, test_mock_sensors.py, and test_multi_mesh_ray_caster.py — these are the three files that use warp APIs without calling wp.init() explicitly.

Important Files Changed

Filename Overview
source/isaaclab/test/utils/test_wrench_composer.py AppLauncher removed; actively uses warp (wp.from_numpy, wp.vec3f, etc.) but lacks explicit wp.init() unlike other kit-less warp tests in the repo
source/isaaclab/test/test_mock_interfaces/test_mock_sensors.py AppLauncher removed; uses wp.to_torch() without explicit wp.init() — relies on warp lazy init, consistent with the change but unlike the existing kit-less warp test pattern
source/isaaclab/test/sensors/test_multi_mesh_ray_caster.py AppLauncher (originally annotated 'Used for warp') removed; imports warp and isaaclab.utils.warp.ops but has no explicit wp.init() call
source/isaaclab/test/utils/test_math.py AppLauncher removed (was needed 'because of warp dependency'); the file now imports only pure-Python/torch/numpy dependencies — no warp transitive pull
source/isaaclab/test/utils/test_configclass.py AppLauncher removed along with the stale NOTE comment about warp; isaaclab.utils.configclass does not import warp, so tests run clean
source/isaaclab/test/terrains/test_terrain_generator.py AppLauncher removed from terrain generation tests; TerrainGenerator may pull in mesh/warp ops internally, but no explicit warp usage in the test file itself
source/isaaclab_contrib/changelog.d/pbarejko-kitless-test-suites.skip Empty .skip file intentionally marks no changelog entry needed for this infrastructure change (same pattern in isaaclab_rl and isaaclab_teleop)
source/isaaclab_rl/test/test_rsl_rl_cfg_deprecation.py AppLauncher removed; test only uses rsl_rl config dataclasses and pytest — no simulation or warp needed
source/isaaclab_teleop/test/test_openxr_device_constructors.py AppLauncher removed; test uses importlib to probe for optional OpenXR device classes — no sim or warp dependency

Comments Outside Diff (1)

  1. source/isaaclab/test/utils/test_wrench_composer.py, line 9 (link)

    P2 Missing wp.init() for kit-less warp usage

    This file imports and actively calls wp.from_numpy(), wp.int32, and wp.vec3f, but unlike other kit-less warp tests in this repo (test_proxy_array.py, test_ray_caster_kernels.py, test_update_ray_caster_kernel.py) that explicitly call wp.config.quiet = True and wp.init(), this test relies on warp's lazy auto-initialization. While warp does auto-initialize in modern releases, the same applies to test_mock_sensors.py (uses wp.to_torch()) and test_multi_mesh_ray_caster.py (uses warp mesh ops) — all three now lack explicit initialization. If the lazy init order ever matters (e.g., when CUDA context is set up by another import first), tests could emit noisy warp startup logs or initialize with the wrong device.

Reviews (1): Last reviewed commit: "Speed test run" | Re-trigger Greptile

@pbarejko pbarejko force-pushed the pbarejko/app-cleanup branch from a6179eb to e6cdd7d Compare May 15, 2026 20:36
@pbarejko pbarejko changed the title Speed test run Speed test run - what happens if we remove Kit startup? May 15, 2026
@pbarejko pbarejko force-pushed the pbarejko/app-cleanup branch from e6cdd7d to 665eb34 Compare May 16, 2026 16:06
@pbarejko pbarejko self-assigned this May 16, 2026
@pbarejko pbarejko changed the title Speed test run - what happens if we remove Kit startup? Test run time improvement: remove AppLauncher from tests that do not require it May 16, 2026
@pbarejko pbarejko requested a review from kellyguo11 May 16, 2026 23:13
@pbarejko pbarejko merged commit f8d167c into isaac-sim:develop May 16, 2026
63 of 64 checks passed
matthewtrepte pushed a commit to matthewtrepte/IsaacLab that referenced this pull request May 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants