Skip to content

[Test Improver] test: add unit tests for RuntimeManager and runtime CLI commands (20%→92%, 49%→90%)#380

Merged
danielmeppiel merged 1 commit intomainfrom
test-assist/runtime-manager-coverage-23324556076-2111a21d57ba6a0f
Mar 22, 2026
Merged

[Test Improver] test: add unit tests for RuntimeManager and runtime CLI commands (20%→92%, 49%→90%)#380
danielmeppiel merged 1 commit intomainfrom
test-assist/runtime-manager-coverage-23324556076-2111a21d57ba6a0f

Conversation

@danielmeppiel
Copy link
Copy Markdown
Collaborator

🤖 Test Improver — automated AI assistant

Goal and Rationale

runtime/manager.py and commands/runtime.py had almost no test coverage (49% and 20% respectively), yet they implement user-facing CLI commands for managing AI runtimes (install/list/remove/status). These are critical-path operations users run when setting up the tool. Low coverage here means regressions go undetected.

Approach

Added tests/unit/test_runtime_manager.py with 53 focused unit tests covering:

RuntimeManager (runtime/manager.py):

  • Initialization: runtime directory, supported runtimes, platform-specific script extensions (.sh vs .ps1)
  • list_runtimes: all-uninstalled, binary in APM dir, binary in system PATH, version detection (success + exception fallback)
  • is_runtime_available: apm dir binary, directory (not file), system PATH fallback, unknown runtime
  • get_available_runtime: priority order, first-wins, none available
  • setup_runtime: unsupported runtime, success/failure/exception, version and vanilla flag forwarding
  • remove_runtime: unknown runtime, copilot npm (success/failure/exception), binary not installed, file removal, directory removal, LLM venv cleanup

commands/runtime CLI (commands/runtime.py):

  • setup: success, failure (exit 1), exception (exit 1), --version, --vanilla, invalid runtime name
  • list: exit 0, calls list_runtimes, exception (exit 1), Rich table fallback path showing runtime names
  • remove: success, failure (exit 1), exception (exit 1), invalid runtime name
  • status: available runtime, no runtime available, exception (exit 1)

Coverage Impact

Module Before After
src/apm_cli/commands/runtime.py 20% 92% (+72pp)
src/apm_cli/runtime/manager.py 49% 90% (+41pp)

Remaining uncovered lines: commands/runtime.py:171-182 (fallback status display with no Rich panel), runtime/manager.py:53-56,85-88,98-101,119-120,137-138,172-174 (PyInstaller bundle paths and Windows-specific script writing — these require platform-specific test environments).

Trade-offs

  • RuntimeManager tests use mocks for subprocess, shutil, and filesystem — fast and hermetic but don't test actual shell script execution (appropriate for unit tests; integration tested in CI scripts)
  • Windows-specific code paths (.ps1 scripts, PowerShell execution) not covered — would need platform-conditional tests or a Windows runner

Reproducibility

uv sync --extra dev
uv run pytest tests/unit/test_runtime_manager.py -v
# Full suite
uv run pytest tests/unit/ --no-header -q

Test Status

✅ All 53 new tests pass
✅ Full suite: 2157 tests pass (up from 2104 before this PR)
✅ No regressions

Generated by Daily Test Improver ·

To install this agentic workflow, run

gh aw add githubnext/agentics/workflows/daily-test-improver.md@b87234850bf9664d198f28a02df0f937d0447295

@danielmeppiel danielmeppiel added automation Deprecated: use type/automation. Kept for issue history; will be removed in milestone 0.10.0. testing Deprecated: use area/testing. Kept for issue history; will be removed in milestone 0.10.0. labels Mar 20, 2026
@danielmeppiel danielmeppiel marked this pull request as ready for review March 22, 2026 15:48
Copilot AI review requested due to automatic review settings March 22, 2026 15:48
@danielmeppiel danielmeppiel force-pushed the test-assist/runtime-manager-coverage-23324556076-2111a21d57ba6a0f branch from 4363fcc to dddbb38 Compare March 22, 2026 15:48
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new unit test suite to substantially increase coverage for the runtime management layer (RuntimeManager) and the user-facing apm runtime CLI commands, reducing regression risk in install/list/remove/status flows.

Changes:

  • Added tests/unit/test_runtime_manager.py covering RuntimeManager behaviors (init, detection, list, setup, remove, embedded script loading).
  • Added Click CliRunner tests for apm runtime setup/list/remove/status success + failure/exception handling.
  • Added targeted coverage for fallback/edge paths (e.g., list fallback output, version detection fallback).

Comment on lines +342 to +346
pass
with pytest.raises(RuntimeError, match="Could not load setup script"):
manager.get_embedded_script("definitely-does-not-exist.sh")


Copy link

Copilot AI Mar 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In test_script_not_found_raises, the with patch("apm_cli.runtime.manager.getattr", return_value=False): pass block is a no-op because it doesn't wrap the call under test. It can be removed, or moved to wrap manager.get_embedded_script(...) if you intended to force the non-PyInstaller path.

Suggested change
pass
with pytest.raises(RuntimeError, match="Could not load setup script"):
manager.get_embedded_script("definitely-does-not-exist.sh")
with pytest.raises(RuntimeError, match="Could not load setup script"):
manager.get_embedded_script("definitely-does-not-exist.sh")

Copilot uses AI. Check for mistakes.
Comment on lines +309 to +342
manager = RuntimeManager()
# Script search walks up from __file__ 4 levels then into scripts/runtime/
# Create a fake script where the code looks for it
current_file = Path(__file__)
# We just check that when a script is found it returns its content
with patch("apm_cli.runtime.manager.Path") as MockPath:
fake_script = MagicMock()
fake_script.exists.return_value = True
fake_script.read_text.return_value = "#!/bin/bash\necho hello"
# Set up the path chain
instance = MagicMock()
instance.__truediv__ = MagicMock(return_value=fake_script)
MockPath.return_value = instance
MockPath.side_effect = None
# Re-create to avoid issues with __init__
# Simpler: patch the actual script path resolution
manager2 = RuntimeManager()
real_script_path = (
Path(__file__).parent.parent.parent
/ "scripts"
/ "runtime"
/ "setup-copilot.sh"
)
if real_script_path.exists():
content = manager2.get_embedded_script("setup-copilot.sh")
assert len(content) > 0
else:
with pytest.raises((FileNotFoundError, RuntimeError)):
manager2.get_embedded_script("nonexistent-script.sh")

def test_script_not_found_raises(self):
manager = RuntimeManager()
with patch("apm_cli.runtime.manager.getattr", return_value=False):
pass
Copy link

Copilot AI Mar 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test_dev_script_found contains an unused tmp_path fixture, unused local variables (manager, current_file), and a with patch("apm_cli.runtime.manager.Path") ... block that never exercises get_embedded_script() (it exits without assertions). This makes the test harder to understand and maintain. Consider removing the dead patch/unused variables and making the test deterministic (e.g., create a temp scripts/runtime tree under tmp_path and patch apm_cli.runtime.manager.__file__ / path resolution so the method reads from that tree).

Suggested change
manager = RuntimeManager()
# Script search walks up from __file__ 4 levels then into scripts/runtime/
# Create a fake script where the code looks for it
current_file = Path(__file__)
# We just check that when a script is found it returns its content
with patch("apm_cli.runtime.manager.Path") as MockPath:
fake_script = MagicMock()
fake_script.exists.return_value = True
fake_script.read_text.return_value = "#!/bin/bash\necho hello"
# Set up the path chain
instance = MagicMock()
instance.__truediv__ = MagicMock(return_value=fake_script)
MockPath.return_value = instance
MockPath.side_effect = None
# Re-create to avoid issues with __init__
# Simpler: patch the actual script path resolution
manager2 = RuntimeManager()
real_script_path = (
Path(__file__).parent.parent.parent
/ "scripts"
/ "runtime"
/ "setup-copilot.sh"
)
if real_script_path.exists():
content = manager2.get_embedded_script("setup-copilot.sh")
assert len(content) > 0
else:
with pytest.raises((FileNotFoundError, RuntimeError)):
manager2.get_embedded_script("nonexistent-script.sh")
def test_script_not_found_raises(self):
manager = RuntimeManager()
with patch("apm_cli.runtime.manager.getattr", return_value=False):
pass
# Build a fake repo layout under tmp_path:
# <repo_root>/scripts/runtime/setup-copilot.sh
repo_root = tmp_path / "repo"
script_dir = repo_root / "scripts" / "runtime"
script_dir.mkdir(parents=True)
script_name = "setup-copilot.sh"
script_path = script_dir / script_name
script_content = "#!/bin/bash\necho hello"
script_path.write_text(script_content)
# RuntimeManager.get_embedded_script walks up from apm_cli.runtime.manager.__file__
# 4 levels to find the repo root, then into scripts/runtime/.
# Create a fake __file__ such that going up 4 parents lands at repo_root.
fake_manager_file = (
repo_root
/ "level1"
/ "level2"
/ "level3"
/ "level4"
/ "manager.py"
)
fake_manager_file.parent.mkdir(parents=True)
fake_manager_file.write_text("# fake manager module")
with patch.object(
sys.modules["apm_cli.runtime.manager"],
"__file__",
str(fake_manager_file),
):
manager = RuntimeManager()
content = manager.get_embedded_script(script_name)
assert content == script_content
def test_script_not_found_raises(self):
manager = RuntimeManager()

Copilot uses AI. Check for mistakes.
…→92%, 49%→90%)

- Add 53 tests covering commands/runtime.py and runtime/manager.py
- RuntimeManager: init, list_runtimes, is_runtime_available, get_available_runtime,
  setup_runtime (success/failure/exception/args), remove_runtime (copilot npm,
  binary file/dir, llm venv cleanup), get_embedded_script
- CLI commands: setup/list/remove/status success and error paths,
  argument validation, Rich table fallback path

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@danielmeppiel danielmeppiel force-pushed the test-assist/runtime-manager-coverage-23324556076-2111a21d57ba6a0f branch from dddbb38 to d0da75c Compare March 22, 2026 16:09
@danielmeppiel danielmeppiel merged commit 4ab99e1 into main Mar 22, 2026
6 checks passed
@danielmeppiel danielmeppiel deleted the test-assist/runtime-manager-coverage-23324556076-2111a21d57ba6a0f branch March 22, 2026 16:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automation Deprecated: use type/automation. Kept for issue history; will be removed in milestone 0.10.0. testing Deprecated: use area/testing. Kept for issue history; will be removed in milestone 0.10.0.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants