apm unpack writes apm.lock.yaml / apm.yml to output dir, violating documented metadata-only contract

[danielmeppiel]

Summary

apm unpack writes apm.lock.yaml and apm.yml into the output directory, even though the CLI's own documentation states these are bundle metadata, not output. This contract violation cascades up the stack:

• microsoft/apm-action's restore mode dumps the lockfile into working-directory (= ${{ github.workspace }} by default) -> dirties tracked files -> any subsequent git checkout fails (see v1.5: restore mode must install apm CLI so bundles are unpacked via 'apm unpack' (not raw tar fallback) apm-action#26).
• github/gh-aw's shared/apm.md template, which uses the action, then crashes any pull_request_target workflow whose triggering PR modifies apm.lock.yaml (see feat(audit): close audit-blindness gap for local .apm/ content (#887) #889).

Originating failure: https://github.com/microsoft/apm/actions/runs/24883083247/job/72856352586?pr=889

error: Your local changes to the following files would be overwritten by checkout:
        apm.lock.yaml
ERR_API: Failed to checkout PR branch: The process '/usr/bin/git' failed with exit code 1

Originating scenario (plain English)

An agentic workflow built on the documented gh-aw template shared/apm.md runs in two phases on a PR:

1. Trusted base phase: checks out main, downloads the APM bundle (built earlier with isolated: 'true' into /tmp/gh-aw/apm-workspace), then calls microsoft/apm-action in restore mode to deploy primitives into the workspace. As a side-effect, the action writes apm.lock.yaml and apm.yml back into the workspace.
2. PR checkout phase: gh-aw runs git checkout -B <branch> origin/pr-head. Git refuses because the workspace is dirty.

When the PR also modifies apm.lock.yaml (which is exactly what every dependency-related PR does), the writes collide and the checkout aborts. The agent never starts.

Root cause: apm unpack contract violation

apm unpack's documented contract states that apm.lock.yaml is bundle metadata and is not copied to the output directory. The implementation does the opposite: it writes the full bundle contents — including apm.lock.yaml and apm.yml — into the output directory. Anything depending on apm unpack (most notably microsoft/apm-action) inherits the bug.

Proposed fix: align CLI with its own contract

Goal: apm unpack <bundle> --output <dir> writes only what the documented contract says it should: agent primitives under .github/{skills,agents,instructions,prompts}/ and apm_modules/. apm.lock.yaml and apm.yml are not written to <dir>.

Behavior change

	Current	Proposed
<dir>/.github/{skills,agents,instructions,prompts}/	Written (overwrites collisions)	Written (no-clobber: existing files in <dir> win, mirroring discover_primitives_with_dependencies priority)
<dir>/apm_modules/	Written	Written (wholesale; gitignored / package-owned)
<dir>/apm.lock.yaml	Written	Not written
<dir>/apm.yml	Written	Not written
Bundle metadata access	Implicit (via written files in <dir>)	Explicit: --metadata-dir <path> flag (optional) writes apm.lock.yaml + apm.yml to a separate caller-controlled path

Why no-clobber for primitives

Today, unpacker.py:37-39 documents: "If a local file has the same name as a bundle file, the bundle file wins (overwrite)." This contradicts APM's own discovery priority (discover_primitives_with_dependencies: local primitives have highest priority). Aligning unpack's collision behavior with discovery's collision behavior makes the model consistent end-to-end.

Why no opt-in flag

Callers who relied on apm.lock.yaml being written to the output dir were relying on undocumented behavior that contradicts the CLI's own contract. Treat as a Fixed bullet in CHANGELOG, not a breaking-flag dance.

Migration

• microsoft/apm-action: build v2 on top of the fixed CLI (v1.5: restore mode must install apm CLI so bundles are unpacked via 'apm unpack' (not raw tar fallback) apm-action#26). v2's restore mode passes through to the fixed apm unpack; workspace stays clean.
• gh-aw shared/apm.md: bump action pin to @v2 once it ships. No template change beyond the version bump (v1.5: restore mode must install apm CLI so bundles are unpacked via 'apm unpack' (not raw tar fallback) apm-action#26 covers the migration plan).
• microsoft/apm own copy of shared/apm.md: keep the temporary post-restore git checkout -- . 2>/dev/null || true workaround until microsoft/apm-action@v2 is available; then drop it and re-import gh-aw's shared/apm.md upstream.
• Direct CLI users invoking apm unpack: those who genuinely need the lockfile post-unpack add --metadata-dir <path>.

Acceptance criteria

• apm unpack <bundle> --output <dir> does not write apm.lock.yaml or apm.yml into <dir>.
• apm unpack <bundle> --output <dir> with a pre-existing tracked primitive (e.g. <dir>/.github/agents/foo.agent.md) does not overwrite it; the local file wins.
• apm unpack <bundle> --output <dir> --metadata-dir <metadir> writes apm.lock.yaml and apm.yml to <metadir> (and not to <dir>).
• Documentation for apm unpack reflects the lockfile-is-metadata contract that the implementation now honors.

Three-layer plan (for cross-link visibility)

1. microsoft/apm (this issue): align apm unpack with its documented contract. Root cause.
2. microsoft/apm-action (v1.5: restore mode must install apm CLI so bundles are unpacked via 'apm unpack' (not raw tar fallback) apm-action#26): ship v2 built on the fixed CLI. working-directory stops getting dirtied on restore. Every action consumer (gh-aw users, raw GH Actions users, anything) gets the fix transparently.
3. github/gh-aw (#28256, closed as redirect): bump shared/apm.md pin to microsoft/apm-action@v2 once available. One-line change. No template gymnastics.

Temporary mitigation (already shipping)

While the CLI + action fixes are in flight, microsoft/apm's own copy of shared/apm.md carries this single-line workaround after the Restore step:

- name: Reset tracked files after APM restore
  run: git checkout -- . 2>/dev/null || true

This is acceptable as a quarantined, single-repo fix because we control the copy. It is not the right shape for upstream gh-aw or for apm-action; those layers get the structural fix above. The workaround becomes a documented no-op once microsoft/apm-action#26 ships.

References

• Originating failing run: https://github.com/microsoft/apm/actions/runs/24883083247/job/72856352586?pr=889
• Triggering PR: feat(audit): close audit-blindness gap for local .apm/ content (#887) #889
• Action-layer follow-up: v1.5: restore mode must install apm CLI so bundles are unpacked via 'apm unpack' (not raw tar fallback) apm-action#26
• gh-aw redirect (closed): shared/apm.md restore step crashes on PRs that modify tracked files (e.g. apm.lock.yaml) github/gh-aw#28256
• Touched paths (mitigation): .github/workflows/shared/apm.md, .github/workflows/*.lock.yml (regenerated), CHANGELOG.md
• Touched paths (root-cause fix): src/apm_cli/... (apm unpack implementation, tests, docs)

Out of scope

• PR-controlled primitive shadowing in pr-review-panel-style workflows (separate higher-severity finding): once this fix lets the workflow succeed, gh-aw's checkout_pr_branch.cjs can replace trusted primitives (e.g. .github/skills/apm-review-panel/SKILL.md) with the PR's versions. Independent of this issue; will file separately against gh-aw with a proper mitigation proposal once the fix chain above lands.

[danielmeppiel]

Filed upstream issue against gh-aw with the proper fix shape: github/gh-aw#28256

Note: the upstream fix recommended there is stronger than the workaround in this issue. Upstream, the recommendation is to extract the bundle into /tmp/gh-aw/apm-restore (symmetric with Pack) and cp -rn-bridge only the new primitives into ${{ github.workspace }}/.github/{skills,agents,instructions,prompts}/ so the consumer's tracked files always win. That eliminates the collision at the root rather than reverting it after the fact.

The workaround proposed in this issue (post-restore git checkout -- .) is the right adopter-side mitigation while we wait for the upstream fix in github/gh-aw#28256, because we control our own copy of shared/apm.md in microsoft/apm. Once gh-aw ships the upstream fix, microsoft/apm can drop its own copy entirely and import shared/apm.md directly from gh-aw, and this workaround becomes a no-op.

[danielmeppiel]

Update: refocused this issue on the CLI root cause and added the three-layer plan. Action-layer follow-up filed at microsoft/apm-action#26. The earlier gh-aw issue (github/gh-aw#28256) was the wrong layer and has been closed with a redirect to here and to microsoft/apm-action#26. The single-line workaround in microsoft/apm's own copy of shared/apm.md stays until microsoft/apm-action@v2 ships, then it becomes a documented no-op.

[danielmeppiel]

Closing -- root cause is in apm-action, not the CLI

After verification (and with thanks to the supply-chain-security expert who pushed back on the original framing), the apm unpack CLI is correct. It only writes deployed_files from the lockfile and never copies apm.lock.yaml / apm.yml to the output dir -- exactly per its documented contract. Verified in src/apm_cli/bundle/unpacker.py.

The actual bug is in microsoft/apm-action's restore mode, which:

1. Deliberately skips installing apm (runner.ts:90 -- "RESTORE MODE: ... skip APM installation entirely").
2. Therefore always falls through to the tar xzf --strip-components=1 fallback in extractBundle (bundler.ts:62-89).
3. Raw tar dumps everything in the bundle into working-directory -- including apm.lock.yaml and apm.yml, which dirties the workspace and breaks any subsequent git checkout.

No CLI change needed. Tracking the proper fix in microsoft/apm-action#26 (ship as v1.5: install apm in restore mode, take the verified apm unpack path; drop the tar-everything fallback as the default).

The temporary git checkout -- . 2>/dev/null || true mitigation in microsoft/apm's own copy of shared/apm.md will land in a small PR (no separate tracker needed) and become a no-op once apm-action@v1.5 is pinned in github/gh-aw shared/apm.md.

References:

• v1.5: restore mode must install apm CLI so bundles are unpacked via 'apm unpack' (not raw tar fallback) apm-action#26 (the actual fix)
• shared/apm.md restore step crashes on PRs that modify tracked files (e.g. apm.lock.yaml) github/gh-aw#28256 (closed; redirected to apm-action#26)
• PR feat(audit): close audit-blindness gap for local .apm/ content (#887) #889 (originating failure)