Skip to content

fix(workflows): gate pr-review-panel on panel-review label at pre-activation#948

Merged
danielmeppiel merged 1 commit intomainfrom
fix/panel-review-label-filter
Apr 26, 2026
Merged

fix(workflows): gate pr-review-panel on panel-review label at pre-activation#948
danielmeppiel merged 1 commit intomainfrom
fix/panel-review-label-filter

Conversation

@danielmeppiel
Copy link
Copy Markdown
Collaborator

Problem

The pr-review-panel workflow was firing on every label change to every PR, not just when the panel-review label was applied.

The label-name guard lived inside the agent prompt as Step 0 ("exit 0 if label != panel-review"), which had two failure modes:

  1. Wasted CI on irrelevant labels. Every label add ran pre_activation + activation + APM bundle restore + agent container spin-up (~50s) before the LLM was even prompted. Then it asked the LLM to please exit early.
  2. Prompt-level instruction is non-deterministic. "Exit 0 if X" is a request the LLM can ignore. Observed: PR [Test Improver] test: add unit tests for policy outcome routing (0% -> ~95%) #943 was labelled automation/testing (never panel-review); the agent ran for 5 min 12 s before stopping. That's a paid LLM call plus full container lifecycle for nothing.

Fix

Move the guard to gh-aw's on.steps: (pre-activation hook). The new label_check step:

on:
  steps:
    - id: label_check
      name: Filter on panel-review label
      run: |
        if [ "$EVENT_NAME" = "workflow_dispatch" ]; then exit 0; fi
        if [ "$LABEL_NAME" = "panel-review" ]; then exit 0; fi
        echo "Triggering label is '$LABEL_NAME' (not 'panel-review'); skipping."
        exit 1
      env:
        EVENT_NAME: ${{ github.event_name }}
        LABEL_NAME: ${{ github.event.label.name }}

When the triggering label is not panel-review, the step exits 1 -> the pre_activation job fails -> all downstream jobs (activation, apm, agent, safe_outputs, etc.) skip. Total cost when filtered out: one ubuntu-slim runner for ~10s. No LLM, no bundle restore, no agent container.

The redundant Step 0 is removed from the prompt body and replaced with a short note pointing at the workflow-level guard.

Why not names:?

gh-aw's names: filter is documented to apply only to pull_request, issues, discussion, and label_command triggers. Adding names: [panel-review] to pull_request_target fails compilation:

Unknown property: names. `names` belongs under `on/label_command`, `on/pull_request`, or `on/issues`

We need pull_request_target to retain secret access on fork PRs (so the agent can post the verdict comment). label_command: would auto-remove the label after activation, but it generates pull_request (not pull_request_target) -- same fork-secret problem. on.steps: is the documented escape hatch.

Validation

  • gh aw compile pr-review-panel succeeds with 0 errors / 0 warnings.
  • Lock file regenerated; label_check step present in pre_activation job.
  • Existing safety posture preserved: read-only permissions, pinned imports to microsoft/apm#main, roles: [admin, maintainer, write], no checkout of PR head.

Test plan after merge

  1. Open any PR with a label other than panel-review -> workflow should run pre_activation only (~10s) and then skip everything else.
  2. Apply panel-review label to a PR -> full panel runs as before.
  3. workflow_dispatch with a pr_number input -> still works (manual override path).

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

…ivation

The label-name guard previously lived inside the agent prompt as Step 0
('exit 0 if label != panel-review'). This had two failure modes:

1. EVERY label add on EVERY PR triggered the workflow, spent ~50s on
   pre_activation + activation + apm bundle restore + agent container
   spin-up, then asked the LLM to please exit early. Observed real
   cost: PR #943 was labelled 'automation'/'testing' (never
   'panel-review') and the agent ran for 5 min 12 s before stopping.

2. The 'exit 0' instruction was a prompt-level request to the LLM,
   not a deterministic gate. An LLM that decides the PR diff is
   interesting can ignore the instruction and proceed.

Fix: move the guard to gh-aw's 'on.steps:' (pre-activation step). When
the triggering label is not 'panel-review' (and the event is not
workflow_dispatch), the step exits 1 -> pre_activation job fails ->
all downstream jobs (activation, apm, agent) skip. Total cost when
filtered out: one ubuntu-slim runner for ~10s, no LLM, no bundle
restore.

gh-aw does not expose 'names:' on pull_request_target (verified at
compile time), so 'on.steps:' is the cheapest available gate. The
prompt's Step 0 is removed; a short note documents that filtering now
happens at the workflow level.

Lock file regenerated via 'gh aw compile pr-review-panel'.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings April 26, 2026 08:10
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes the pr-review-panel agentic workflow so it only runs the expensive panel pipeline when the triggering label is panel-review, rather than on every label change. It does this by moving the label-name guard from the LLM prompt into gh-aw's on.steps: pre-activation hook, preventing APM restore and the agent container from starting on irrelevant labels.

Changes:

  • Add a deterministic pre-activation label_check step under on.steps: to gate execution to panel-review (or allow manual workflow_dispatch).
  • Remove the prompt-embedded Step 0 label guard, replacing it with a short note pointing to the workflow-level gate.
  • Regenerate the compiled workflow lock file to reflect the new pre-activation step and updated prompt content.
Show a summary per file
File Description
.github/workflows/pr-review-panel.md Adds on.steps label gate and removes the prompt-level label guard instructions.
.github/workflows/pr-review-panel.lock.yml Updates the compiled workflow to inject the label_check step into pre_activation and refreshes prompt/metadata hashes.

Copilot's findings

  • Files reviewed: 2/2 changed files
  • Comments generated: 0

@danielmeppiel danielmeppiel merged commit e928d63 into main Apr 26, 2026
18 checks passed
@danielmeppiel danielmeppiel deleted the fix/panel-review-label-filter branch April 26, 2026 08:24
@danielmeppiel danielmeppiel mentioned this pull request Apr 26, 2026
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants