Review Retrospective — PR #664
PR: fix: pass drift data via step outputs — agent can't access activation filesystem
Merged: 2026-04-22T03:57Z by @PureWeen
Review runs: 0 automated reviews completed
What Happened
The review-on-open.agent triggered on PR open (03:34 UTC) but the PR was merged at 03:57 UTC — 23 minutes later. The review workflow requires 3 parallel sub-agents with adversarial consensus and has a 90-minute timeout. It could not complete in that window. Zero reviews or comments were posted before merge.
This was the third consecutive workflow PR (#653 → #662 → #664) iterating on the same instruction-drift workflow. Each went in without a completed automated review.
Skill Coverage
| Skill |
Relevant? |
Referenced? |
Gap? |
gh-aw-guide |
✅ Yes — workflow files changed |
❌ Review did not run |
⚠️ Structural timing gap |
instruction-drift |
✅ Yes — the skill being fixed |
❌ Review did not run |
⚠️ Structural timing gap |
processing-state-safety |
❌ No C# code changed |
— |
— |
| All other skills |
❌ Not applicable |
— |
— |
Review Accuracy
| Metric |
Count |
| True Positives |
0 |
| True Positives (Fixed) |
0 |
| False Positives |
0 |
| False Negatives |
0 (review never ran) |
| Unresolved |
1 (pre-existing, see below) |
PR Changes Were Correct
The diff was sound:
- Lock file properly regenerated (
frontmatter_hash updated, verified via GH_AW_STEPS_* env vars added to all agent invocation steps)
issues: write removed from frontmatter is safe — compiler adds it to safe_outputs and conclusion jobs automatically; agent job only needs contents: read + pull-requests: read$GITHUB_OUTPUT template substitution pattern is correct- Python setup step correctly added for
python3 JSON parsing in activation job
Pre-Existing Gap (Not Introduced by This PR)
create-pull-request in instruction-drift.agent.md lacks github-token-for-extra-empty-commit: — per the gh-aw anti-patterns guide, GITHUB_TOKEN pushes do not trigger CI on agent-created PRs. Drift PRs currently land with no CI run. This pre-dates PR #664 and was not caught by any prior review.
Improvement Suggestions
-
Add status-comment: true to review-on-open.agent.md — posts a start/complete comment when the workflow triggers, signaling to the PR author that automated review is in progress. Without it there's no indicator that the 90-minute review was even triggered, so authors routinely merge before it finishes.
-
Branch protection: Consider requiring the Expert Code Review (auto) status check to pass before merging to main. This is the only robust solution to the timing gap for fast-merged infra/workflow PRs.
-
Fix github-token-for-extra-empty-commit on instruction-drift.agent.md's create-pull-request safe output — a PAT or App token is required to trigger CI on generated drift PRs. See the gh-aw anti-patterns table.
Generated by Review Retrospective for issue #664 · ◷
Review Retrospective — PR #664
PR: fix: pass drift data via step outputs — agent can't access activation filesystem
Merged: 2026-04-22T03:57Z by @PureWeen
Review runs: 0 automated reviews completed
What Happened
The
review-on-open.agenttriggered on PR open (03:34 UTC) but the PR was merged at 03:57 UTC — 23 minutes later. The review workflow requires 3 parallel sub-agents with adversarial consensus and has a 90-minute timeout. It could not complete in that window. Zero reviews or comments were posted before merge.This was the third consecutive workflow PR (#653 → #662 → #664) iterating on the same instruction-drift workflow. Each went in without a completed automated review.
Skill Coverage
gh-aw-guideinstruction-driftprocessing-state-safetyReview Accuracy
PR Changes Were Correct
The diff was sound:
frontmatter_hashupdated, verified viaGH_AW_STEPS_*env vars added to all agent invocation steps)issues: writeremoved from frontmatter is safe — compiler adds it tosafe_outputsandconclusionjobs automatically; agent job only needscontents: read+pull-requests: read$GITHUB_OUTPUTtemplate substitution pattern is correctpython3JSON parsing in activation jobPre-Existing Gap (Not Introduced by This PR)
create-pull-requestininstruction-drift.agent.mdlacksgithub-token-for-extra-empty-commit:— per the gh-aw anti-patterns guide,GITHUB_TOKENpushes do not trigger CI on agent-created PRs. Drift PRs currently land with no CI run. This pre-dates PR #664 and was not caught by any prior review.Improvement Suggestions
Add
status-comment: truetoreview-on-open.agent.md— posts a start/complete comment when the workflow triggers, signaling to the PR author that automated review is in progress. Without it there's no indicator that the 90-minute review was even triggered, so authors routinely merge before it finishes.Branch protection: Consider requiring the
Expert Code Review (auto)status check to pass before merging tomain. This is the only robust solution to the timing gap for fast-merged infra/workflow PRs.Fix
github-token-for-extra-empty-commitoninstruction-drift.agent.md'screate-pull-requestsafe output — a PAT or App token is required to trigger CI on generated drift PRs. See the gh-aw anti-patterns table.