Skip to content

bug: quality gate dispatched before Copilot review completes #178

@microsasa

Description

@microsasa

Problem

The orchestrator dispatches the quality gate as soon as it sees CI green + 0 open threads. But Copilot review may not have completed yet — it runs asynchronously after CI. This means:

  1. Quality gate evaluates and approves the PR
  2. Copilot review finishes and adds new comments
  3. The APPROVE review is now stale — it was based on pre-review state
  4. Auto-merge may proceed despite unresolved threads (mitigated by required_conversation_resolution: true in branch protection, but the stale approval persists)

Example

PR #172: quality gate dispatched at 07:23 while Copilot was still reviewing. QG approved, then Copilot posted a new comment at 07:25. The approval was based on stale state.

Impact

  • Wasted quality gate runs (approves then new comments appear)
  • Stale APPROVE reviews persist because dismiss_stale_reviews: false
  • Could lead to merges with unaddressed review comments if branch protection is ever relaxed

Fix options

  1. Orchestrator checks for Copilot review before dispatching QG — verify copilot-pull-request-reviewer has submitted a review on the latest commit
  2. Quality gate agent checks for pending reviews — look at whether Copilot review is complete before evaluating
  3. Dismiss stale reviews on new pushes — set dismiss_stale_reviews: true so responder pushes invalidate the QG approval (but this affects all reviews, not just QG)

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions