Skip to content

Refactor: L3 SceneTestCase — HostCallable as Python DAG#514

Merged
ChaoWao merged 1 commit intohw-native-sys:mainfrom
hw-native-sys-bot:refactor/l3-scene-test
Apr 10, 2026
Merged

Refactor: L3 SceneTestCase — HostCallable as Python DAG#514
ChaoWao merged 1 commit intohw-native-sys:mainfrom
hw-native-sys-bot:refactor/l3-scene-test

Conversation

@hw-native-sys-bot
Copy link
Copy Markdown
Collaborator

Summary

  • Extend SceneTestCase to support L3 distributed tests (multi-chip + SubWorker)
  • L3 CALLABLE structure mirrors L2: orchestration is a Python DAG function, callables is a mixed list of ChipCallable entries (compiled like L2) and SubCallable entries (registered as callable_id)
  • Add CallableNamespace for orch functions to access compiled callables by name with dot-access, with keep() for lifetime management past drain()
  • Extract _compile_chip_callable_from_spec helper reused by both L2 and L3 paths
  • Rewrite test_l3_dependency.py and test_l3_group.py as SceneTestCase subclasses with module-level orch + sub functions
  • Update conftest.py L3 fixture to register SubCallable entries and read device_count/num_sub_workers from case config dict

Test plan

  • pip install . builds successfully
  • pytest tests/ut -x -q — 149 passed
  • L2 regression: pytest examples/.../test_vector_example.py --platform a2a3sim — PASSED
  • L3 manual test: Python direct execution of full L3 flow — f[0]=47.0 (correct)
  • CI: L3 sim tests via pytest

🤖 Generated with Claude Code

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for Level 3 (L3) scene tests, enabling orchestration via Python DAG functions. Key additions include the CallableNamespace for dot-access to callables and lifetime management, refactored path resolution and compilation logic to support multiple callables, and updated test runners to handle L3 worker initialization. Feedback addresses a potential name collision in the standalone runner's shared sub_ids dictionary, risks of attribute overwriting in CallableNamespace due to direct dict updates, and a potential IndexError in the keep method when invoked without arguments.

@hw-native-sys-bot hw-native-sys-bot force-pushed the refactor/l3-scene-test branch 4 times, most recently from b0eb626 to 8a95a98 Compare April 10, 2026 09:41
…e + SubCallable

Extend SceneTestCase to support L3 distributed tests (multi-chip + SubWorker).

L3 CALLABLE mirrors L2 structurally:
- orchestration: Python DAG function (vs C++ binary for L2)
- callables: list of ChipCallable entries (compiled) + SubCallable entries
  (registered), distinguished by field presence

Key additions:
- CallableNamespace: dot-access container for orch functions to access
  compiled ChipCallables and registered SubCallable IDs by name, with
  keep() for lifetime management of transient objects past drain()
- _compile_chip_callable_from_spec: extracted from compile_chip_callable,
  reused by both L2 and L3 compilation paths
- _run_and_validate_l3: builds CallableNamespace, wraps orch in Task,
  compares all tensors against golden
- conftest.py L3 branch: registers SubCallable entries from CALLABLE,
  reads device_count/num_sub_workers from case config dict
- Rewrites test_l3_dependency.py and test_l3_group.py as SceneTestCase
  subclasses with module-level orch + sub functions

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@ChaoWao ChaoWao merged commit 4b28dcd into hw-native-sys:main Apr 10, 2026
12 of 13 checks passed
@ChaoWao ChaoWao deleted the refactor/l3-scene-test branch April 10, 2026 11:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants