fix(compile): emit and clean up copilot root instructions (#792)

[danielmeppiel]

fix(compile): emit and clean up copilot root instructions (#792)

Note

Supersedes #930 (original author @WilliamK112 preserved at 59c5136c). Rebased cleanly on current main (4 conflict files including a 150-line region in agents_compiler.py) and extended with the first real-world consumer of the new compile path. Validation evidence on the original PR: #930 (comment).

TL;DR

Closes #792 by teaching apm compile to emit .github/copilot-instructions.md from the global (no-applyTo) instructions in .apm/instructions/, with idempotent regeneration, build-id markers, and safe cleanup of stale generated files (manually-authored files are preserved). Scope attestation: this PR is copilot-target-only. Other harness root files (.cursorrules, Aider, Windsurf) are intentionally untouched; root AGENTS.md is the natural next step and remains tracked under #695.

Problem (WHY)

Per the issue, APM the package manager already compiles .apm/instructions/ into per-pattern Copilot files, but the root Copilot file is hand-authored — exactly the format-fragmentation pain APM was built to solve, and visible to anyone reading the repo.

• Dogfood gap. The repo "hand-authors .github/copilot-instructions.md (26 lines, top-level Copilot pointer with project philosophy, review-panel directive, dev-iteration commands, etc.) — NOT compiled, must be maintained by hand" while every other instruction surface is generated.
• No compile target for root prose. The issue's gap table lists .github/copilot-instructions.md as the one row marked "GAP — file this issue".
• Drift risk. Without compile coverage, hand-edits to the root file silently diverge from .apm/instructions/*.md — the same multi-format drift the README promises to eliminate.
• [!] Acceptance criteria unmet. The issue requires the compile to be "idempotent and round-trips cleanly" with "at least one fixture proving the emission, one proving round-trip stability".

Approach (WHAT)

#	Fix
1	Add _maybe_emit_copilot_root_instructions in agents_compiler.py: synthesizes .github/copilot-instructions.md from the global (no-applyTo) instructions in the primitive collection.
2	Gate emission on should_compile_copilot_instructions_md(target) in target_detection.py so only Copilot-routed targets produce the root file.
3	Stamp every generated file with <!-- Generated by APM CLI from .apm/ primitives --> plus a deterministic <!-- Build ID: ... --> line so cleanup can distinguish generated from hand-authored files.
4	Cleanup path: when the target is no longer Copilot-routed, OR when no global instructions remain, remove only files carrying the generated marker; manually-authored files are preserved.
5	Fold minimal semantics into the new code path so -t minimal stays AGENTS.md-only and does not implicitly behave like vscode.
6	First real-world consumer: ship .apm/instructions/linting.instructions.md as the canonical lint contract and let the new path regenerate .github/copilot-instructions.md from it.

Implementation (HOW)

• src/apm_cli/compilation/agents_compiler.py — adds _maybe_emit_copilot_root_instructions, _generate_copilot_root_instructions_content, _finalize_build_id, _cleanup_copilot_root_instructions (~150 lines). Threaded after both the distributed and single-file branches of compile() so the Copilot root output works under either AGENTS.md strategy. New imports: hashlib, BUILD_ID_PLACEHOLDER, should_compile_copilot_instructions_md.
• src/apm_cli/core/target_detection.py — adds should_compile_copilot_instructions_md(target) predicate (the single gate for the new path).
• src/apm_cli/integration/targets.py — adds generated_files field to target metadata so future targets can declare their own root-prose surfaces without re-touching the compiler.
• src/apm_cli/commands/compile/cli.py — drops the minimal -> vscode mapping inside the multi-target frozenset branch so -t minimal retains its semantics.
• .apm/instructions/linting.instructions.md — NEW. Portable lint contract (no applyTo: lint is a workflow gate, not a per-file rule). Source of truth for the lint contract across Copilot, Claude Code, Cursor, Codex.
• .apm/skills/pr-description-skill/SKILL.md + .github/skills/pr-description-skill/SKILL.md — re-point the defense-in-depth lint gate at the new portable file instead of the Copilot-specific copilot-instructions.md. Closes the portability hole where the skill ships across harnesses but referenced a Copilot-only file.
• .github/copilot-instructions.md — regenerated by apm compile -t copilot; the previous manually-maintained Linting block is gone, replaced by the synthesized output (carries the <!-- Build ID: ... --> marker).
• Tests — 4 files, 198 PR-added tests: tests/integration/test_compile_copilot_root_instructions.py, tests/unit/compilation/test_compile_target_flag.py, tests/unit/core/test_target_detection.py, tests/unit/integration/test_targets.py.

Diagrams

Legend: data flow showing how apm compile -t copilot routes .apm/instructions/*.md to its three Copilot-target output surfaces. The dashed node is the NEW surface introduced by this PR; the other two surfaces are pre-existing.

flowchart LR
    subgraph Source[".apm/instructions/"]
        S1["linting.instructions.md (global, no applyTo)"]
        S2["python.instructions.md (applyTo: **/*.py)"]
        S3["other *.instructions.md"]
    end
    subgraph Compile["apm compile -t copilot"]
        C1["distributed / single-file pipeline"]
        C2["per-pattern integrator"]
        C3["_maybe_emit_copilot_root_instructions"]:::new
    end
    subgraph Output["generated outputs"]
        O1["AGENTS.md"]
        O2[".github/instructions/*.instructions.md"]
        O3[".github/copilot-instructions.md (NEW root prose)"]:::new
    end
    S1 --> C1 --> O1
    S2 --> C2 --> O2
    S3 --> C2
    S1 --> C3 --> O3
    classDef new stroke-dasharray: 5 5,stroke-width:2px;

Loading

Legend: cleanup behaviour when a developer re-runs compile against a non-Copilot target. The compiler removes only files it recognises as generated (build-id marker present); a hand-authored file at the same path is preserved untouched.

sequenceDiagram
    participant Dev as Developer
    participant CLI as "apm compile -t non-copilot"
    participant Gate as should_compile_copilot_instructions_md
    participant FS as .github/copilot-instructions.md
    Dev->>CLI: invoke
    CLI->>Gate: target eligible?
    Gate-->>CLI: false
    CLI->>FS: read existing file
    alt file contains generated marker
        CLI->>FS: unlink (stats.removed = 1)
        Note over FS: stale generated file cleaned up
    else manually-authored (no marker)
        Note over FS: preserved untouched
    end

Loading

Trade-offs

• Copilot-only scope. This PR ships the root-prose synthesis for the copilot target only. Cursor (.cursorrules), Aider (.aider.conf.yml), Windsurf, and other harness root files are intentionally NOT emitted by this code path. Rationale: keep the surface area small, prove the pattern on the highest-share Copilot target first, then extend per harness in follow-up PRs once [FEATURE] Create root AGENTS.md documenting all CI-enforced rules for coding agents #695 (root AGENTS.md) is also resolved. The generated_files field on target metadata is the seam future targets plug into without re-touching the compiler.
• Marker-based cleanup over content hashing. Chose a <!-- Build ID: ... --> HTML-comment marker; rejected content-hash matching because it cannot tell a hand-edit of generated content apart from a freshly authored file. The marker is unambiguous and inert to Copilot's reader.
• Global instructions only (no aggregation of applyTo-scoped files). The root file synthesizes from .apm/instructions/*.md that have no applyTo: — scoped instructions continue to flow into per-pattern .github/instructions/*.instructions.md mirrors (existing path). Avoids double-emission.
• minimal stays AGENTS.md-only. Resisted folding minimal into the Copilot-routed set; that would have broken the explicit minimal-mode contract.

Benefits

1. The README's "write-once, run-anywhere" claim is now provable on this repo's own root Copilot file: one .apm/instructions/linting.instructions.md edit lands in .github/copilot-instructions.md on the next apm compile -t copilot.
2. 198 PR-added tests pin emission, cleanup, marker preservation, and minimal-mode semantics — all green.
3. Removing the hand-authored Linting block deletes a drift vector: the lint contract is now sourced from one portable file consumed by every harness.
4. The generated_files target-metadata seam unblocks [FEATURE] Create root AGENTS.md documenting all CI-enforced rules for coding agents #695 (root AGENTS.md) and future Cursor / Windsurf root-file work without further compiler surgery.
5. Manually-authored .github/copilot-instructions.md files remain safe — cleanup is gated on the generated marker.

Validation

uv run --extra dev ruff check src/ tests/:

All checks passed!

uv run --extra dev ruff format --check src/ tests/:

604 files already formatted

uv run apm compile -t copilot (after deleting the manually-authored file):

[+] Compilation completed successfully!
[!]   .apm/instructions/linting.instructions.md: No 'applyTo' pattern specified -- instruction will apply globally

Lint contract confirmed flowing from the new portable instruction:

$ head -6 .github/copilot-instructions.md
<!-- Generated by APM CLI from .apm/ primitives -->
<!-- Build ID: 3a5991d7ca52 -->
<!-- APM Version: 0.11.0 -->

<!-- Source: .apm/instructions/linting.instructions.md -->
# Linting (canonical contract)

$ grep -A 2 "CI .Lint. job runs:" .github/copilot-instructions.md
The CI `Lint` job runs:

- `uv run --extra dev ruff check src/ tests/`
- `uv run --extra dev ruff format --check src/ tests/`

Full test output (PR-added + unit suite)

PR-added tests (4 files):

$ PYTHONPATH=src uv run pytest tests/unit/compilation/test_compile_target_flag.py \
    tests/integration/test_compile_copilot_root_instructions.py \
    tests/unit/core/test_target_detection.py \
    tests/unit/integration/test_targets.py -x
198 passed in 2.71s

Unit suite (matches CI; tests/unit/test_audit_report.py skipped per project convention for its pre-existing Py3.11 SyntaxError):

$ uv run pytest tests/unit tests/test_console.py -x
6848 passed, 1 warning, 30 subtests passed in 40.24s

Cross-link: full rebase + conflict-resolution narrative on the superseded PR — #930 (comment).

How to test

• Check out the branch: git fetch origin issue-792-copilot-root-instructions && git switch issue-792-copilot-root-instructions, then uv sync --extra dev.
• Run the four PR-added test files: PYTHONPATH=src uv run pytest tests/unit/compilation/test_compile_target_flag.py tests/integration/test_compile_copilot_root_instructions.py tests/unit/core/test_target_detection.py tests/unit/integration/test_targets.py -x — expect 198 passed.
• Delete .github/copilot-instructions.md, run uv run apm compile -t copilot, then git diff — expect the file to be regenerated with a <!-- Build ID: ... --> line and the lint contract sourced from .apm/instructions/linting.instructions.md.
• Run uv run apm compile -t claude and uv run apm compile -t codex against a fresh checkout — expect NO .github/copilot-instructions.md to be created (copilot-only attestation).
• Hand-author a .github/copilot-instructions.md (no marker), run uv run apm compile -t claude — expect the file to be preserved untouched (cleanup is marker-gated).

Follow-ups

• [FEATURE] Create root AGENTS.md documenting all CI-enforced rules for coding agents #695 (root AGENTS.md) is the natural next step; this PR's generated_files seam is the plug-in point.
• A subsequent PR on microsoft/apm-action could add a --check-style drift gate so CI fails when .apm/instructions/* is edited but compiled outputs are not regenerated.
• Cursor (.cursorrules), Aider (.aider.conf.yml), Windsurf root files: same pattern, one PR per harness.

Co-authored-by: WilliamK112 164879897+WilliamK112@users.noreply.github.com
Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

[Copilot]

Pull request overview

Adds support for apm compile to generate the Copilot root instructions file (.github/copilot-instructions.md) from global (non-applyTo) instruction primitives, including deterministic build markers and marker-gated cleanup of stale generated files.

Changes:

• Emit and update .github/copilot-instructions.md during compile for Copilot-capable targets, and clean it up when not applicable.
• Extend target metadata and target descriptions to include the new generated root file.
• Add unit + integration tests plus a new canonical linting instruction primitive used as a real consumer of the root-instructions pipeline.

Show a summary per file

File	Description
src/apm_cli/compilation/agents_compiler.py	Implements Copilot root instructions generation, build-id stamping, and cleanup logic.
src/apm_cli/core/target_detection.py	Adds routing predicate and updates target descriptions to mention the new output.
src/apm_cli/integration/targets.py	Adds generated_files to target metadata and populates it for Copilot.
src/apm_cli/commands/compile/cli.py	Preserves minimal semantics by no longer mapping it to vscode.
tests/integration/test_compile_copilot_root_instructions.py	Verifies emission + idempotence and stale-file cleanup behavior end-to-end.
tests/unit/compilation/test_compile_target_flag.py	Unit-tests root emission/cleanup and minimal behavior.
tests/unit/core/test_target_detection.py	Unit-tests new routing predicate and updated target descriptions.
tests/unit/integration/test_targets.py	Asserts Copilot profile metadata includes the new generated file.
.apm/instructions/linting.instructions.md	Adds portable canonical lint contract as a global instruction primitive.
.apm/skills/pr-description-skill/SKILL.md	Re-points the skill to the new lint-contract primitive (portable across harnesses).
.github/skills/pr-description-skill/SKILL.md	Mirrors the same reference update for the compiled skill surface.
.github/copilot-instructions.md	Regenerated output committed for Copilot root instructions.

Copilot's findings

• Files reviewed: 12/12 changed files
• Comments generated: 5

[Copilot]

In multi-target mode (config.target is a frozenset), _maybe_emit_copilot_root_instructions() sets routing_target to that frozenset and then calls should_compile_copilot_instructions_md(), which only returns True for string targets. This means apm compile -t claude,copilot (or any multi-target list including Copilot) will never emit .github/copilot-instructions.md, even when .github/ exists and Copilot was explicitly requested.

Consider handling the frozenset case explicitly (e.g., emit when the workspace is Copilot-routed by presence of .github/, or plumb the original target tokens so you can distinguish copilot from other agents-family targets in mixed-family runs).

[Copilot]

The footer line *To regenerate: specify apm compile* reads like a placeholder and is not an actionable command for contributors. Since this text is emitted into .github/copilot-instructions.md, please replace it with the real regeneration command (for example, apm compile -t copilot or the exact invocation intended for this repo).

Suggested change

	sections.append("*To regenerate: `specify apm compile`*")
	sections.append("*To regenerate: `apm compile -t copilot`*")

[Copilot]

The minimal target description now suggests creating only .github/, .claude/, or .gemini/ for full integration, but detect_target() and the rest of the code support additional target roots (.cursor/, .opencode/, .codex/, etc.). This string is user-facing and should either be generic ("create a target folder") or list all supported roots to avoid misleading guidance.

Suggested change

	"minimal": "AGENTS.md only (create .github/, .claude/, or .gemini/ for full integration)",
	"minimal": "AGENTS.md only (create a supported target folder for full integration)",

[Copilot]

There is an unmatched closing parenthesis in binding live in .apm/instructions/linting.instructions.md). which makes the sentence read incorrectly. Remove the stray ) (and optionally adjust to "binding lives in ..." for grammar).

Suggested change

	binding live in `.apm/instructions/linting.instructions.md`). If lint is red,
	binding lives in `.apm/instructions/linting.instructions.md`. If lint is red,

[Copilot]

Same issue here: binding live in .apm/instructions/linting.instructions.md). has an unmatched closing parenthesis. Remove the stray ) so the instruction reads cleanly.

Suggested change

	binding live in `.apm/instructions/linting.instructions.md`). If lint is red,
	binding live in `.apm/instructions/linting.instructions.md`. If lint is red,