-
Notifications
You must be signed in to change notification settings - Fork 0
fix: quality-gate accepts COMMENTED reviews from Copilot #80
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -31,10 +31,10 @@ Audit/Health Agent → creates issue (max 2) → dispatches Implementer | |
| → Implementer creates PR (lint-clean, non-draft, auto-merge, aw label) | ||
| → CI runs + Copilot auto-reviews (parallel, via ruleset) | ||
| → CI fails? → CI Fixer agent (1 retry, label guard) | ||
| → Copilot has comments? → Review Responder addresses them (1 attempt, label guard) | ||
| → Copilot approves → Quality Gate evaluates quality + blast radius | ||
| → Copilot has comments? → Review Responder addresses them (pushes fixes) | ||
| → Copilot reviews (COMMENTED state) → Quality Gate evaluates quality + blast radius | ||
|
Owner
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Good catch. Updated the pipeline diagram to read
|
||
| → LOW/MEDIUM impact → approves → auto-merge fires | ||
| → HIGH impact → flags for human review | ||
| → HIGH impact → flags for human review (auto-merge stays blocked) | ||
| ``` | ||
|
|
||
| </details> | ||
|
|
@@ -291,6 +291,8 @@ EOF | |
| - **Reviewer identity**: `copilot-pull-request-reviewer[bot]` (login: `copilot-pull-request-reviewer`) | ||
| - **Event actor**: `Copilot` (the GitHub App identity — this is what `context.actor` returns and what `check_membership.cjs` matches against) | ||
|
|
||
| > **Pitfall**: Copilot auto-reviews almost always submit as `COMMENTED`, not `APPROVED`. Any downstream workflow that triggers on `pull_request_review` and checks the review state must accept `COMMENTED` reviews from Copilot — not just `APPROVED`. The Quality Gate handles this correctly. | ||
|
|
||
| ### Addressing Copilot review comments (GraphQL) | ||
|
|
||
| ```bash | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -4,6 +4,14 @@ Append-only history of repo-level changes (CI, infra, shared config). Tool-speci | |
|
|
||
| --- | ||
|
|
||
| ## fix: Quality Gate trigger condition — accept COMMENTED reviews from Copilot — 2026-03-15 | ||
|
|
||
| **Problem**: Quality Gate instructions required the triggering review to be an APPROVAL from Copilot. But Copilot auto-reviews almost always submit as `COMMENTED` (not `APPROVED`), so the Quality Gate would see the COMMENTED state and stop immediately (noop). This meant the Quality Gate never actually evaluated or approved agent PRs, and auto-merge stayed blocked. | ||
|
|
||
| **Fix**: Updated Quality Gate instructions to accept both COMMENTED and APPROVED reviews from Copilot. Added documentation about the auto-merge flow: Quality Gate approval is what triggers GitHub auto-merge on agent PRs. (PR #80, closes #81) | ||
|
Owner
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This has already been updated — line 11 now reads
|
||
|
|
||
| --- | ||
|
|
||
| ## fix: Copilot actor name mismatch in bots list — 2026-03-15 | ||
|
|
||
| **Problem**: `check_membership.cjs` matches `context.actor` (`Copilot`) against `GH_AW_ALLOWED_BOTS`, but bots list only had `copilot-pull-request-reviewer` (the reviewer login). Actor name mismatch → `activated = false` → agent jobs skipped. | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -16,6 +16,7 @@ | |
| import click | ||
| from rich.console import Console | ||
| from rich.table import Table | ||
| from watchdog.events import FileSystemEventHandler # type: ignore[import-untyped] | ||
| from watchdog.observers import Observer # type: ignore[import-untyped] | ||
|
|
||
| from copilot_usage.models import SessionSummary | ||
|
|
@@ -114,14 +115,15 @@ def _read_line_nonblocking(timeout: float = 0.5) -> str | None: | |
| return None | ||
|
|
||
|
|
||
| class _FileChangeHandler: | ||
| class _FileChangeHandler(FileSystemEventHandler): # type: ignore[misc] | ||
| """Watchdog handler that triggers refresh on any session-state change.""" | ||
|
|
||
| def __init__(self, change_event: threading.Event) -> None: | ||
| super().__init__() | ||
| self._change_event = change_event | ||
|
Comment on lines
+118
to
123
Owner
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Valid observation. The CLI change (
|
||
| self._last_trigger = 0.0 | ||
|
|
||
| def dispatch(self, event: object) -> None: # noqa: ANN001 | ||
| def dispatch(self, event: object) -> None: | ||
| now = time.monotonic() | ||
| if now - self._last_trigger > 2.0: # debounce 2s | ||
| self._last_trigger = now | ||
|
|
@@ -132,7 +134,7 @@ def _start_observer(session_path: Path, change_event: threading.Event) -> object | |
| """Start a watchdog observer monitoring *session_path* for changes.""" | ||
| handler = _FileChangeHandler(change_event) | ||
| observer = Observer() | ||
| observer.schedule(handler, str(session_path), recursive=True) # type: ignore[arg-type] | ||
| observer.schedule(handler, str(session_path), recursive=True) | ||
| observer.daemon = True | ||
| observer.start() | ||
| return observer | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed — the stop condition was ambiguous. Updated it to: "If the triggering review is not from Copilot, or the review state is not COMMENTED or APPROVED, stop immediately." This now matches the preceding bullets. Fixed in commit ebf1bae.