fix(planner/tests): scope test_advisory_mode stubs so they don't leak#8418
Conversation
test_advisory_mode.py injected stubs for dynamo._core, dynamo.runtime, dynamo.llm, and dynamo.common.forward_pass_metrics into sys.modules at module-import (collection) time. Because the file is collected before the other planner/profiler tests, those later modules resolved against the stubs and hit ImportError on real attributes (ModelType, QueuedRequestMetrics), tripping the `pytest.skip(...)` guards added in the same PR. Result on the planner-test-cpu image: 12 extra skips, only 151 passed vs 335 before. Move the injection into a scope="module", autouse=True fixture that skips modules already imported (don't shadow reals) and uses MonkeyPatch().setitem + undo() so sys.modules is clean after this module's tests finish. Sibling tests now see the real modules. Verified against the c388483 planner-test-cpu image with the CI pytest command: 356 passed, 1 skipped, 31 deselected (up from 153 / 13 / 11). The single remaining skip is the legitimate test_virtual_connector.py guard. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: Dmitry Tokarev <dtokarev@nvidia.com>
WalkthroughA test module's dependency stubbing was refactored from global Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes 🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
🧹 Nitpick comments (1)
components/src/dynamo/planner/tests/unit/test_advisory_mode.py (1)
23-25: Hoist fixture-local imports to the top of the file.
sys,types, andunittest.mock.MagicMockare standard-library imports with no optional-dependency concern, so they should live at module top per the repo Python guideline. Unlike the lazyPlannerConfigimports inside test methods (which must be deferred until this fixture installs stubs), these have no such constraint.♻️ Proposed refactor
+import sys +import types +from unittest.mock import MagicMock + import pytest from dynamo.planner.config.defaults import SLAPlannerDefaults @@ `@pytest.fixture`(scope="module", autouse=True) def _stub_heavy_deps(): # Stub optional Rust/IO-heavy dynamo modules when absent so planner.config # imports used below can resolve. Scoped to this module and torn down after # so sibling test modules see the real modules (regression: `#8244` left # these stubs in sys.modules and caused later tests to skip). - import sys - import types - from unittest.mock import MagicMock - stubs = {As per coding guidelines (
.ai/python-guidelines.md): "keep all imports at the file top (no imports inside functions/classes) and avoid import-hiding patterns … unless it's explicitly for optional dependencies".🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@components/src/dynamo/planner/tests/unit/test_advisory_mode.py` around lines 23 - 25, Move the standard-library imports sys, types and the unittest.mock import MagicMock out of any fixture- or function-local scope and place them at the top of the module; locate the fixture or test functions where sys, types, or MagicMock are currently imported (e.g., the imports referenced as sys, types, MagicMock) and hoist those three import statements to module-level, leaving deferred imports like PlannerConfig inside tests untouched.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@components/src/dynamo/planner/tests/unit/test_advisory_mode.py`:
- Around line 23-25: Move the standard-library imports sys, types and the
unittest.mock import MagicMock out of any fixture- or function-local scope and
place them at the top of the module; locate the fixture or test functions where
sys, types, or MagicMock are currently imported (e.g., the imports referenced as
sys, types, MagicMock) and hoist those three import statements to module-level,
leaving deferred imports like PlannerConfig inside tests untouched.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 11d1108d-8a74-474e-aa83-ce17df1e8ffa
📒 Files selected for processing (1)
components/src/dynamo/planner/tests/unit/test_advisory_mode.py
…mConnector TestRefreshWorkerInfoFromConnector._make_planner builds a NativePlannerBase with environment="kubernetes", which instantiates KubernetesConnector -> KubernetesAPI -> config.load_kube_config() and registers Prometheus Enum metrics globally. The tests never ran on main because the sys.modules leak from test_advisory_mode.py skipped this whole module; unmasking the tests exposed two pre-existing issues: - No kube config / DYN_PARENT_DGD_K8S_NAME in the container. - Per-test NativePlannerBase instantiation double-registers the `dynamo_planner_load_scaling_decision` Enum in the global CollectorRegistry. Patch KubernetesAPI, PlannerPrometheusMetrics, and inject DYN_PARENT_DGD_K8S_NAME in `_make_planner` to sandbox these. Verified locally with the CI pytest command: `392 passed, 1 skipped, 74 deselected`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: Dmitry Tokarev <dtokarev@nvidia.com>
With the stub-leak + K8s/Prometheus mocking fixes in this PR, the planner CPU tests run cleanly under pytest-xdist. Drop the explicit sequential override so both pr.yaml planner-test and post-merge-ci.yml planner-pipeline fall back to `auto` (the shared-test.yml default). Local timings on the planner-test-cpu image (392 tests): -n 0 73s -n 2 58s -n 4 62s -n auto 61s Runner core count isn't guaranteed, so `auto` is the right pick over a hard-coded count. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: Dmitry Tokarev <dtokarev@nvidia.com>
CI runners expose 8 cores and `-n auto` runs ~110s because the per- worker startup/isolation overhead dominates for a 392-test suite with mostly small tests. Two workers consistently beat higher counts in local measurement: -n 0 73s -n 2 58s ← best -n 4 62s -n auto 61s (laptop, 8 cores CI ~110s) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: Dmitry Tokarev <dtokarev@nvidia.com>
Summary
test_advisory_mode.pyinstalled stubs fordynamo._core,dynamo.runtime,dynamo.llm, anddynamo.common.forward_pass_metricsintosys.modulesat module-import (pytest collection) time. Because it's collected alphabetically before the other planner/profiler tests, those later modules resolved their imports against the stubs — stubs that don't exposeModelType,QueuedRequestMetrics, etc. — which tripped thepytest.skip(...)guards added in the same PR (#8244).Net effect on the
dynamo-planner-test-cpuimage with the CI command:71af9675): 335 passed, 1 skippedc388483a): 153 passed, 13 skippedFix: move the stub injection into a
scope="module", autouse=Truefixture thatpytest.MonkeyPatch().setitem(sys.modules, ...),mp.undo()on teardown sosys.modulesis clean for sibling test modules.Unmasking those tests also surfaced two pre-existing issues in
TestRefreshWorkerInfoFromConnector(added in #8042, never exercised): the test builtNativePlannerBasewithenvironment="kubernetes", which callsconfig.load_kube_config()and registers PrometheusEnummetrics globally. PatchedKubernetesAPI,PlannerPrometheusMetrics, andDYN_PARENT_DGD_K8S_NAMEinside_make_plannerto sandbox them.pytest parallelism
CI runners expose 8 cores, but
-n autois slow because per-worker startup/isolation dominates for a 392-test suite. Measured on this PR's planner-test image (392 passed, 1 skipped on every run):-n 0-n 2-n 4-n auto(8)Switched planner's CI override to
cpu_parallel_mode: '2'for both pre-merge and post-merge. Two workers matchautoon CI and beat it locally;-n 0is ~50% slower.Verification
Local run of the exact CI pytest command against the
c388483aplanner-test-cpu image with this fix + theTestRefreshWorkerInfoFromConnectormocks:vs. same image unpatched:
153 passed, 13 skipped, 11 deselected. The single remaining skip is the legitimatetest_virtual_connector.py:48guard.Test plan
planner-teston amd64 and arm64 reports 392 passed / 1 skipped.test_advisory_mode.pyortest_load_based_scaling.py.-n 2.🤖 Generated with Claude Code