[review-retro] Review Retrospective — PR #662

[github-actions]

PR: fix: run drift check scripts in steps: where gh CLI is authenticated
Merged: 2026-04-22T03:25:25Z by PureWeen
Review runs: 0 automated reviews (PR self-merged ~5 min after opening; review-on-open triggered but could not complete in time)

---

Skill Coverage Analysis

Skill	Relevant?	Referenced?	Gap?
gh-aw-guide	✅ Yes (all changed files are workflow .md/.lock.yml)	❌ No review ran	⚠️ Would have caught unauthenticated gh CLI in retro prompt
processing-state-safety	❌ No (no C# files changed)	—	—
copilot-sdk-reference	❌ No	—	—
performance-optimization	❌ No	—	—
multi-agent-orchestration	❌ No	—	—

---

Review Accuracy

Metric	Count
True Positives	0
True Positives (Fixed)	0
False Positives	0
False Negatives (missed)	1
Unresolved	1

---

Missed Findings (False Negatives)

🟡 MODERATE — review-retro.agent.md Step 1 uses unauthenticated gh CLI (same bug class PR #662 was fixing)

File: .github/workflows/review-retro.agent.md, lines 57–72

Issue: Step 1 instructs the agent to gather PR data via:

gh pr view ${{ github.event.pull_request.number }} --json ...
gh api repos/.../pulls/.../reviews --jq ...
gh api repos/.../issues/.../comments --jq ...
gh pr diff ${{ github.event.pull_request.number }}

These gh CLI calls require GH_TOKEN, which is scrubbed inside the agent container. This is the exact same bug that PR #662 was fixing in instruction-drift.agent.md (where Check-Staleness.ps1 called gh api and gh issue view). The fix for instruction-drift was to move script execution to steps: where GH_TOKEN is available. The newly-added review-retro.agent.md was added in the same PR with the same bug.

Confirmed workaround: The compiled lock file (review-retro.agent.lock.yml:550) shows GitHub MCP tools ARE provisioned by default: "GITHUB_TOOLSETS": "context,repos,issues,pull_requests". An agent that ignores the gh CLI instructions and uses MCP tools instead will work correctly. The current run succeeded this way. But an agent that faithfully executes the Step 1 bash commands as written will receive authentication errors and may fail to gather any PR data.

Fix: Rewrite review-retro.agent.md Step 1 to use GitHub MCP tools instead of gh CLI:

• get_pull_request → replaces gh pr view
• get_pull_request_reviews → replaces gh api .../reviews
• list_pull_request_comments → replaces gh api .../comments
• list_issue_comments → replaces gh api .../issues/.../comments
• get_pull_request_diff → replaces gh pr diff
• list_pull_request_files → replaces gh pr diff --name-only

Also add a note: > Do NOT use gh CLI to gather data — gh is not authenticated inside the agent container. Use GitHub MCP tools exclusively.

Skill that would have caught this: gh-aw-guide — the skill documents that gh CLI credentials are scrubbed in the agent container and that MCP tools or steps: are the correct alternative.

---

Process Finding

PR #662 was self-merged 5 minutes after opening (created 03:20, merged 03:25 UTC). The review-on-open.agent.md workflow likely triggered on PR open but the expert review takes 20–90 minutes. This left the PR — which added three new workflow files (fix.agent.md, review-retro.agent.md, verify-build.yml) — without any automated or human review before merge.

The unreviewed review-retro.agent.md then triggered this retrospective run, which is how the false negative was discovered.

---

Improvement Suggestions

1. Fix review-retro.agent.md Step 1 — Replace all gh CLI data-gathering commands with equivalent GitHub MCP tool calls. Add an explicit note that gh is not authenticated in the agent container. This matches the pattern already used by review-shared.md and fix-shared.md.

2. Consider adding a steps: pre-computation block to review-retro.agent.md — Similar to the pattern in the fixed instruction-drift.agent.md. Alternatively, since GitHub MCP tools work correctly (confirmed by the lock file), simply updating the instructions to use them is sufficient and simpler.

3. Workflow self-review gap — When a PR adds or modifies gh-aw workflow .md files, consider adding a checklist reminder: "Did the new workflow instructions avoid unauthenticated gh CLI calls?" This is a recurring error class in this repo (PR fix: run drift check scripts in steps: where gh CLI is authenticated #662 fixed it once, introduced it again in the same PR).

---

This retrospective was generated automatically by the review-retro workflow for PR #662.

Warning

⚠️ Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• patchdiff.githubusercontent.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "patchdiff.githubusercontent.com"

See Network Configuration for more information.

Generated by Review Retrospective for issue #662 · ◷

• expires on May 22, 2026, 3:39 AM UTC

[github-actions]

This issue is being closed as outdated. A newer issue has been created: #671

View newer issue

---

This action was performed automatically by the Review Retrospective workflow.