bug: apm run start picks wrong runtime — system PATH stub takes priority over APM-managed binary, codex v0.116+ incompatible with GitHub Models

[edenfunf]

Description

apm run start selects the wrong runtime in two separate but related scenarios:

Scenario A — System PATH stub wins over APM-managed binary

When a broken runtime stub exists in the system PATH (e.g. a GitHub CLI extension registers a copilot shim), _detect_installed_runtime finds it via shutil.which() before checking ~/.apm/runtimes/. The selected binary may be non-functional.

Scenario B — codex v0.116+ is incompatible with GitHub Models

codex removed wire_api = "chat" support in v0.116 and now requires wire_api = "responses" (OpenAI Responses API). GitHub Models only exposes the Chat Completions endpoint — the /responses endpoint returns 404. There is no configuration workaround; the two are fundamentally incompatible at the API level.

apm runtime setup codex installs the latest release (currently v0.117), which cannot work with GitHub Models regardless of config.

Steps to Reproduce

Scenario A:

1. Install codex via apm runtime setup codex
2. Have any other tool registered as copilot in PATH (e.g. gh extension install github/gh-copilot)
3. apm run start — copilot stub is selected, codex is ignored

Scenario B:

1. apm runtime setup codex (installs v0.116+)
2. $env:GITHUB_TOKEN = "..."; apm run start
3. Observe:

Error loading config.toml: `wire_api = "chat"` is no longer supported.

Changing to wire_api = "responses":

ERROR: unexpected status 404 Not Found, url: https://models.github.ai/inference/responses

Expected Behaviour

• APM-installed binaries in ~/.apm/runtimes/ take priority over anything in PATH.
• apm runtime setup codex either installs a version that works with GitHub Models, or clearly documents the incompatibility and suggests an alternative.

Environment

• OS: Windows 11
• apm version: v0.8.11
• codex version: 0.117.0 (rust-based CLI)

[github-actions]

Triage decision

accept

Proposed labels

area/cli
area/docs-site
type/bug
status/accepted
priority/high

Milestone

0.9.4

Suggested next action

Fix Scenario A by making _detect_installed_runtime prefer ~/.apm/runtimes/ over PATH; fix Scenario B by pinning apm runtime setup codex to a known-compatible codex version or emitting a clear compatibility error message, and document the codex/GitHub-Models incompatibility in the runtime setup docs.

Suggested issue comment

Both scenarios are confirmed bugs -- thanks for the detailed reproduction and environment info.

**Scenario A (PATH precedence)**: `_detect_installed_runtime` should check `~/.apm/runtimes/` before falling back to `shutil.which()`. APM-managed binaries must always take priority over system PATH entries, both for correctness and to prevent PATH-injection attacks where a stub overrides a managed binary.

**Scenario B (codex v0.116+ incompatibility)**: `apm runtime setup codex` installs the latest version, which as of v0.116 requires the OpenAI Responses API (`wire_api = "responses"`). GitHub Models only exposes the Chat Completions endpoint -- the two are fundamentally incompatible at the API level. The fix options are:
1. Pin `apm runtime setup codex` to the last known-compatible version (e.g., v0.115.x) until GitHub Models adds Responses API support.
2. Emit a clear error at setup time: "codex v0.116+ is incompatible with GitHub Models; install v0.115.x manually or use an alternative runtime."

Option 2 is the safer long-term approach (avoids silently pinning a stale version). Either way, the incompatibility must be documented in the runtime setup docs.

Starting points:
- Runtime detection: [src/apm_cli/run/](https://github.com/microsoft/apm/tree/main/src/apm_cli/run/)
- Runtime setup command: [src/apm_cli/commands/runtime.py](https://github.com/microsoft/apm/tree/main/src/apm_cli/commands/runtime.py)

Per-lens notes (collapsed)

DevX UX Expert -- User-Need Reviewer

Two distinct but related bugs in the apm run flow. Scenario A breaks the core promise that APM manages its own runtimes -- if a PATH stub silently wins, the user's APM-installed runtime is ignored with no error or warning. Scenario B is a silent failure: apm runtime setup codex succeeds, but the installed codex version cannot work with GitHub Models. Both damage the core apm run start user experience (P0 severity).

Supply Chain Security Expert -- Risk-Surface Reviewer

Scenario A has a secondary security implication: PATH-based stub override could be exploited to substitute a malicious binary for the APM-managed runtime (PATH injection). Preferring ~/.apm/runtimes/ over PATH closes this attack vector. area/cli applies. No broad theme/security flag -- this is standard PATH resolution hygiene rather than a systemic supply chain surface.

OSS Growth Hacker -- Contributor-Tone Reviewer

Not activated -- author is a CONTRIBUTOR with established project history; standard technical bug-fix discussion tone is appropriate.

Python Architect -- Architecture Reviewer

Not activated -- both fixes are well-scoped. Scenario A: modify _detect_installed_runtime to check ~/.apm/runtimes/ before shutil.which() (a few lines). Scenario B: update the codex setup logic to pin a compatible version or emit an explicit error (a few lines + a compatibility check). No cross-cutting design decisions required.

Doc Writer -- Documentation Reviewer

Activated -- Scenario B (codex incompatibility with GitHub Models) must be documented in the runtime setup docs: either a compatibility table or a note in the codex runtime setup section explaining that codex v0.116+ requires the Responses API, which GitHub Models does not support. area/docs-site added as secondary area.

APM CEO -- Triage Arbiter

Two P0 bugs in apm run start -- accept with priority/high. Both are well-scoped fixes. No theme label needed -- pure CLI infra. area/cli and area/docs-site (for the Scenario B compatibility note). Milestone: 0.9.4 for the next patch release. The fix for Scenario B should include a docs update documenting the codex/GitHub-Models incompatibility so users can make informed decisions.

{
  "decision": "accept",
  "decision_detail": "",
  "theme": null,
  "areas": ["area/cli", "area/docs-site"],
  "type": "type/bug",
  "status": "status/accepted",
  "priority": "priority/high",
  "preserved_labels": [],
  "milestone": "0.9.4",
  "next_action": "Fix _detect_installed_runtime to prefer ~/.apm/runtimes/ over PATH (Scenario A); fix codex setup to pin a compatible version or emit a clear error for v0.116+ incompatibility with GitHub Models, and document the incompatibility (Scenario B).",
  "comment_markdown": "Both scenarios confirmed. Scenario A: check ~/.apm/runtimes/ before shutil.which(). Scenario B: pin codex to last-compatible version or emit explicit error at setup time; document the codex v0.116+/GitHub-Models incompatibility in runtime setup docs."
}

---

Triage status: agentic proposal pending human ratification.
Silence is approval. Maintainers can:

• Override any label or milestone above by editing it directly --
  human edits are authoritative and will not be reverted on
  subsequent runs.
• Re-trigger triage by applying the status/needs-triage label, or
  by removing status/triaged to enroll the issue in the next
  daily sweep.

Posted by the Triage Panel workflow. See .apm/skills/apm-triage-panel for the panel skill.

Generated by Triage Panel · ● 1.6M · ◷