fix: quality-gate accepts COMMENTED reviews from Copilot by microsasa · Pull Request #80 · microsasa/cli-tools

microsasa · 2026-03-15T18:01:56Z

Problem

Quality Gate instructions required the triggering Copilot review to be an APPROVED state. But Copilot auto-reviews almost always submit as COMMENTED (inline comments, workflow-only PRs, etc.), so the Quality Gate would see the COMMENTED state and stop immediately (noop).

This meant:

Quality Gate never evaluated agent PRs
No approval was ever submitted
Auto-merge stayed blocked forever

Closes #81

Fix

Updated Quality Gate instructions to accept both COMMENTED and APPROVED reviews from Copilot
Clarified the auto-merge flow in instructions (Quality Gate approval triggers GitHub auto-merge)
Added pitfall documentation about Copilot review states
Updated changelog

bug: agent workflows never activate on Copilot review events #75 — Previous activation bug (different: agents weren't activating at all)
Autonomous agent pipeline — CI fixer, review responder, quality gate #53 — Original pipeline spec
PR fix: use roles: all workaround for bot activation (gh-aw#21098) #76 — Fixed activation, exposed this bug

Testing

Next agent PR after merge — Quality Gate should actually evaluate and approve LOW/MEDIUM impact, triggering auto-merge.

- _FileChangeHandler now inherits from watchdog.events.FileSystemEventHandler - Added super().__init__() call in __init__ - Removed # noqa: ANN001 suppression from dispatch - Removed # type: ignore[arg-type] from observer.schedule() - Added test verifying inheritance and dispatch behavior Closes #67 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot auto-reviews submit as COMMENTED, not APPROVED. The quality-gate instructions required APPROVAL, so it would noop on every Copilot review and never approve agent PRs — blocking auto-merge. Updated instructions to accept COMMENTED reviews from Copilot. Added documentation about the auto-merge flow and the Copilot review state pitfall. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Updates the agentic workflow documentation/instructions so the Quality Gate can run when Copilot submits a COMMENTED (not only APPROVED) review, unblocking autonomous approval/auto-merge for agent PRs.

Changes:

Adjust Quality Gate instructions to proceed on Copilot reviews in COMMENTED or APPROVED state.
Clarify the auto-merge flow and document the Copilot review-state pitfall.
Update changelog and add a small watchdog handler typing/test adjustment in the CLI.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
`tests/copilot_usage/test_cli.py`	Adds a test asserting the watchdog handler inherits from `FileSystemEventHandler`.
`src/copilot_usage/cli.py`	Makes `_FileChangeHandler` subclass `FileSystemEventHandler` and removes a type-ignore on `observer.schedule`.
`docs/changelog.md`	Adds a changelog entry describing the Quality Gate trigger-condition fix.
`docs/agentic-workflows.md`	Updates the pipeline diagram and adds a note about Copilot review states.
`.github/workflows/quality-gate.md`	Updates Quality Gate instructions to accept `COMMENTED` Copilot reviews and clarifies auto-merge behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

microsasa · 2026-03-15T18:10:44Z

docs/changelog.md

+
+**Problem**: Quality Gate instructions required the triggering review to be an APPROVAL from Copilot. But Copilot auto-reviews almost always submit as `COMMENTED` (not `APPROVED`), so the Quality Gate would see the COMMENTED state and stop immediately (noop). This meant the Quality Gate never actually evaluated or approved agent PRs, and auto-merge stayed blocked.
+
+**Fix**: Updated Quality Gate instructions to accept both COMMENTED and APPROVED reviews from Copilot. Added documentation about the auto-merge flow: Quality Gate approval is what triggers GitHub auto-merge on agent PRs. (PR #80, closes #81)


This has already been updated — line 11 now reads (PR #80, closes #81). No placeholder remains.

Generated by Review Responder for issue #80

microsasa · 2026-03-15T18:10:47Z

docs/agentic-workflows.md

-      → Copilot has comments? → Review Responder addresses them (1 attempt, label guard)
-      → Copilot approves → Quality Gate evaluates quality + blast radius
+      → Copilot has comments? → Review Responder addresses them (pushes fixes)
+      → Copilot reviews (COMMENTED state) → Quality Gate evaluates quality + blast radius


Good catch. Updated the pipeline diagram to read COMMENTED or APPROVED state, usually COMMENTED so it accurately reflects that both states trigger Quality Gate. Fixed in commit ebf1bae.

Generated by Review Responder for issue #80

microsasa · 2026-03-15T18:10:46Z

.github/workflows/quality-gate.md

   - The reviewer is `copilot-pull-request-reviewer` (the Copilot reviewer bot)
-   If the triggering review is not a Copilot approval, stop immediately.
+   - The review state is COMMENTED or APPROVED (Copilot auto-reviews submit as COMMENTED, not APPROVED)
+   If the triggering review is not from Copilot, stop immediately.


Agreed — the stop condition was ambiguous. Updated it to: "If the triggering review is not from Copilot, or the review state is not COMMENTED or APPROVED, stop immediately." This now matches the preceding bullets. Fixed in commit ebf1bae.

Generated by Review Responder for issue #80

microsasa · 2026-03-15T18:10:45Z

src/copilot_usage/cli.py

+class _FileChangeHandler(FileSystemEventHandler):  # type: ignore[misc]
    """Watchdog handler that triggers refresh on any session-state change."""

    def __init__(self, change_event: threading.Event) -> None:
+        super().__init__()
        self._change_event = change_event


Valid observation. The CLI change (_FileChangeHandler subclassing FileSystemEventHandler + removing type: ignore) is a small typing fix that was bundled since it's low-risk and already had a test added. That said, the PR description should have called this out more clearly. I'll leave this for the PR author to decide whether to split — it's a scoping/process question rather than a code fix I should make unilaterally.

Generated by Review Responder for issue #80

github-actions

Quality Gate: APPROVED ✅

Evaluated dimensions:

Code quality: Good — changes are well-structured, proper FileSystemEventHandler inheritance replaces duck-typing, type-ignore comments cleaned up, meaningful test added.
Blast radius: MEDIUM — Primarily workflow instruction and documentation updates fixing a real pipeline bug (Quality Gate never activated because Copilot reviews submit as COMMENTED, not APPROVED). The CLI change is a small typing/inheritance fix with test coverage.

Copilot review notes: 4 comments were raised, all minor documentation wording suggestions. No bugs, security issues, or logic errors flagged.

Medium-impact pipeline fix with good documentation and test coverage. Auto-approving for merge.

microsasa added the aw Created by agentic workflow label Mar 15, 2026

Copilot AI review requested due to automatic review settings March 15, 2026 18:01

Copilot started reviewing on behalf of microsasa March 15, 2026 18:02 View session

microsasa force-pushed the fix/quality-gate-accept-commented-reviews branch from 27c4d01 to a8d67e8 Compare March 15, 2026 18:03

Copilot AI reviewed Mar 15, 2026

View reviewed changes

github-actions bot approved these changes Mar 15, 2026

View reviewed changes

microsasa added the review-response-attempted label Mar 15, 2026

microsasa merged commit 9342776 into main Mar 15, 2026
16 of 21 checks passed

microsasa deleted the fix/quality-gate-accept-commented-reviews branch March 15, 2026 18:11

This was referenced Mar 15, 2026

bug: admin merge pattern race condition allows unapproved auto-merges #83

Open

[aw] Review Responder failed #82

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: quality-gate accepts COMMENTED reviews from Copilot#80

fix: quality-gate accepts COMMENTED reviews from Copilot#80
microsasa merged 2 commits intomainfrom
fix/quality-gate-accept-commented-reviews

microsasa commented Mar 15, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

microsasa Mar 15, 2026

Uh oh!

microsasa Mar 15, 2026

Uh oh!

microsasa Mar 15, 2026

Uh oh!

microsasa Mar 15, 2026

Uh oh!

github-actions bot left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants


		Problem: Quality Gate instructions required the triggering review to be an APPROVAL from Copilot. But Copilot auto-reviews almost always submit as `COMMENTED` (not `APPROVED`), so the Quality Gate would see the COMMENTED state and stop immediately (noop). This meant the Quality Gate never actually evaluated or approved agent PRs, and auto-merge stayed blocked.

		Fix: Updated Quality Gate instructions to accept both COMMENTED and APPROVED reviews from Copilot. Added documentation about the auto-merge flow: Quality Gate approval is what triggers GitHub auto-merge on agent PRs. (PR #80, closes #81)

Conversation

microsasa commented Mar 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Fix

Related

Testing

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

microsasa Mar 15, 2026

Choose a reason for hiding this comment

Uh oh!

microsasa Mar 15, 2026

Choose a reason for hiding this comment

Uh oh!

microsasa Mar 15, 2026

Choose a reason for hiding this comment

Uh oh!

microsasa Mar 15, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

microsasa commented Mar 15, 2026 •

edited

Loading