[review-retro] Review Retrospective — PR #743 (chore: 3 review rounds for agent-fix)

[github-actions]

Review Retrospective — PR #743

PR: chore: 3 review rounds for agent-fix
Merged: 2026-04-23 by PureWeen
Review runs: 0 automated reviews

---

Skill Coverage Analysis

Skill	Relevant?	Referenced?	Gap?
gh-aw-guide	✅ Yes (workflow .md changed)	❌ No review ran	⚠️ Should have been consulted
processing-state-safety	❌ No (no CopilotService changes)	—	—
multi-agent-orchestration	❌ No	—	—
performance-optimization	❌ No	—	—
copilot-sdk-reference	❌ No	—	—

---

Review Accuracy

Metric	Count
True Positives	0
True Positives (Fixed)	0
False Positives	0
False Negatives (missed)	1
Unresolved	1

---

No Review Ran

The review-on-open.agent.md workflow is configured to trigger on pull_request: [opened, ready_for_review] for non-draft PRs. PR #743 was opened, non-draft, and merged — but zero review runs, zero comments, and zero inline feedback were recorded. The automated expert review did not execute.

---

Missed Findings (False Negatives)

🔴 CRITICAL: Contradictory round-limit instructions in agent-fix.md

File: .github/workflows/agent-fix.md
Introduced by: PR #743 (the very PR this retro covers)

The PR updated Step 7 (line 166) to read:

Repeat Steps 6-7 up to 3 times (max 3 review rounds).

But Rule 4 (line 207) was not updated and still reads:

4. Max 2 review rounds. After 2 rounds, post remaining findings as PR comments.

An agent running agent-fix.md will now receive contradictory instructions: the procedural steps say stop at 3, but the rules section says stop at 2. Agents typically obey the rule they see most recently or the more conservative one, making the actual behavior non-deterministic.

Fix needed: Update Rule 4 in .github/workflows/agent-fix.md to:

4. Max 3 review rounds. After 3 rounds, post remaining findings as PR comments.

Which skill would have caught it: The gh-aw-guide skill instructs reviewers to check workflow .md files for internal consistency. A reviewer consulting that skill would have verified that all references to round limits are consistent.

---

Root Cause

The review-on-open workflow did not execute. Without the automated review, no model checked internal consistency of the edited workflow file, and the contradictory rule slipped through.

---

Improvement Suggestions

1. Fix the inconsistency now — update Rule 4 in agent-fix.md from "Max 2" to "Max 3" review rounds.
2. Add a consistency check to the review prompt — when a workflow .md file is changed, the reviewer should grep for all numeric references to the same concept (e.g., "round", "max") and verify they agree.
3. Investigate why review-on-open didn't run — check the workflow run history for PR chore: 3 review rounds for agent-fix #743. If the workflow was skipped or failed to queue, that's a reliability gap in the review pipeline.

---

/label retrospective
expires: 30
close-older-issues: true

Generated by Review Retrospective for issue #743 · ◷

• expires on May 23, 2026, 3:37 AM UTC

[github-actions]

This issue is being closed as outdated. A newer issue has been created: #752

View newer issue

---

This action was performed automatically by the Review Retrospective workflow.