-
Notifications
You must be signed in to change notification settings - Fork 0
Description
Problem
When Copilot submits a pull_request_review, the review-responder and quality-gate workflows trigger but the agent never runs. pre_activation completes with success but activated output is false, causing activation and agent jobs to be skipped.
Root Cause
This is caused by a bug in gh-aw's check_membership.cjs (github/gh-aw#21098). The role check calls getCollaboratorPermissionLevel("Copilot") which returns a 404 (Copilot is a GitHub App, not a user). This error causes an early return before the GH_AW_ALLOWED_BOTS fallback is ever evaluated.
The bots: field compiles correctly into the lock file, but the runtime code never checks it.
Fix
Use roles: all as a workaround. This tells the compiler to skip the permission check entirely (needsRoleCheck returns false), so check_membership.cjs is not included in the pre_activation job. The activated output defaults to true and the agent runs.
This is overly permissive but acceptable as a temporary workaround. See #74 to track removing it when gh-aw fixes the upstream bug.
Previous Attempts (all failed)
Each of these was merged to main based on incomplete understanding:
-
PR fix: allow Copilot reviewer bot to trigger review-responder and quality-gate #64 / Issue Fix incorrect bots: placement from PR #64 #70 — Added
bots:at top level +roles: all. Wrong becausebots:at top level is silently ignored by the compiler. Theroles: allwould have actually worked (we now know), but we didn't understand why at the time and reverted it. -
PR fix: correct bots: placement under on: for agent activation #65 — Moved
bots:underon:(correct placement). Fixed the compilation issue but didn't fix activation because of the upstreamcheck_membership.cjsbug. -
PR fix: add Copilot actor to bots list for agent activation #72 / Issue bug: Copilot actor name mismatch blocks agent activation #73 — Added
Copilotto the bots list (the event actor name). Correct identification of the actor name, but irrelevant becausecheck_membership.cjsnever reaches the bot check due to the error branch.
What the Fix PR Will Do
- Add
roles: allto review-responder.md and quality-gate.md - Keep
bots: [Copilot, copilot-pull-request-reviewer](already present, harmless, will be needed when upstream is fixed) - Recompile lock files
- Update docs to reflect the workaround and history
Related
- Upstream bug: github/gh-aw#21098
- Tracking removal of workaround: tracking: gh-aw bot allowlist bug (github/gh-aw#21098) #74
- Previous attempts: PR fix: allow Copilot reviewer bot to trigger review-responder and quality-gate #64 (issue Fix incorrect bots: placement from PR #64 #70), PR fix: correct bots: placement under on: for agent activation #65, PR fix: add Copilot actor to bots list for agent activation #72 (issue bug: Copilot actor name mismatch blocks agent activation #73)