docs: add integrity filtering, idempotency, trigger guide to gh-aw skill#754
docs: add integrity filtering, idempotency, trigger guide to gh-aw skill#754
Conversation
Comprehensive update incorporating validated security patterns: - Integrity filtering: full hierarchy (merged > approved > unapproved > none > blocked), interaction matrix with roles:, configuration examples including trusted-users, blocked-users, and approval-labels - Idempotency requirement for slash commands + lock-for-agent guidance - Edited-comment time-bomb: issue_comment.edited fires TODAY with today's secrets on 6-month-old comments — use types: [created] by default - Read-only contributor write surface: what read-only users can fire, defense priority list - Trigger selection guide: recommended/caution/avoid tiers with rationale - Design principles: deterministic-by-default, limitations-are-security, least-privilege, signal-to-noise - Triage role footgun: label_command needs triage but default roles excludes it - Pre-cancellation race: already-running steps complete after cancel All claims validated against official GitHub docs and gh-aw reference. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Security Documentation Review —
|
There was a problem hiding this comment.
Expert Code Review — PR #754
docs: add integrity filtering, idempotency, trigger guide to gh-aw skill
Methodology
3 independent reviewers conducted parallel deep reviews, followed by adversarial consensus with targeted follow-up verification on disputed findings. Findings required ≥2/3 agreement to be included.
Summary of Findings
| # | Severity | Finding | Agreement |
|---|---|---|---|
| 1 | 🔴 CRITICAL | min-integrity: approved shown as active config directly contradicts the compiler-bug crash warning that follows it — copy-paste hazard |
3/3 |
| 2 | 🟡 MODERATE | roles: YAML indentation as sibling of slash_command: may silently disable access control |
2/3 |
| 3 | 🟡 MODERATE | Fork guard YAML gotcha suggests non-standard parentheses fix; should use canonical ${{ }} |
2/3 |
| 4 | 🟡 MODERATE | roles: all "acceptable" in Trigger Guide contradicts "NEVER" in Security Pattern #1 |
2/3 |
| 5 | 🟡 MODERATE | pull_request in ⛔ Avoid (unqualified) contradicts existing Noise Reduction section showing valid pull_request + paths: patterns |
3/3 |
| 6 | 🟡 MODERATE | Cancellation signal sequence omits SIGINT and misstates timing — actual is SIGINT → 7500ms → SIGTERM → 2500ms → kill |
3/3 |
Discarded Findings (single-reviewer only, no consensus)
- Edited-comment time-bomb mitigation missing for
slash_command:(1/3) roles:auto-injection claim unverified (1/3)- Fork guard only gates
jobs.guard, not other jobs (1/3)
Overall Assessment
This is a high-quality, security-focused documentation update that adds significant value — the integrity filtering hierarchy, trigger selection guide, and edited-comment time-bomb warning are all important additions. The critical finding (Finding 1) should be addressed before merge: the min-integrity example currently inverts the safety guidance from the previous version, creating a copy-paste hazard that crashes the MCP Gateway. The remaining moderate findings are internal consistency issues that could mislead developers relying on this as a security reference.
CI / Review Status
- CI: Commit status pending (no status checks reported); activation and pre_activation checks passed
- Prior reviews: None
- Changed files: 1 file (
.claude/skills/gh-aw-guide/SKILL.md, +135/-17)
Generated by Expert Code Review (auto) for issue #754 · ● 9.4M
| tools: | ||
| github: | ||
| min-integrity: approved # Default for public repos — only trusted author content | ||
| toolsets: [pull_requests, repos] |
There was a problem hiding this comment.
🔴 CRITICAL — min-integrity example contradicts its own crash warning (Flagged by: 3/3 reviewers)
This line shows min-integrity: approved as an uncommented, active configuration value with an inline comment presenting it as the default. However, the warning at line ~274 states that hardcoded min-integrity in source "emits an incomplete guard policy (missing repos field) that crashes the MCP Gateway."
The previous version of this file unambiguously marked this as ❌ BROKEN with the value commented out. The new version inverts the guidance — the example looks like the recommended configuration, with a softened footnote caveat appearing after the code block. Anyone copy-pasting this YAML gets a workflow that crashes the MCP Gateway at startup.
Suggested fix: Either comment out min-integrity: approved (as was done previously), or move the compiler-bug warning above the YAML block with a clear ❌ marker so the danger is visible before the copyable code.
| name: review | ||
| events: [pull_request_comment] | ||
| roles: [admin, maintainer, write] # Only committers can trigger — NEVER use 'all' unless you've audited every safe-output | ||
| ``` |
There was a problem hiding this comment.
🟡 MODERATE — roles: YAML nesting level may silently disable access control (Flagged by: 2/3 reviewers)
roles: is indented as a sibling of slash_command: (a key under on:), not as a sub-field of slash_command:. If the gh-aw schema expects roles: under slash_command:, this placement would make it silently ignored — meaning the workflow has no access control while appearing to have it.
Concrete failure: A developer copies this pattern verbatim. If roles: is ignored at this nesting level, any user (including the PR author) can trigger /review on a malicious PR — the exact scenario this pattern is supposed to prevent.
Suggested fix: Verify the correct schema level for roles: and adjust indentation accordingly. If it's valid at the on: level, add an explicit note confirming this, since events: being under slash_command: creates the natural expectation that roles: also belongs there.
| ``` | ||
|
|
||
| > ⚠️ **YAML gotcha**: Don't start a bare `if:` value with `!` — it's a YAML tag indicator. Either wrap in `${{ }}` or use parentheses: `if: (!github.event.repository.fork)`. | ||
|
|
There was a problem hiding this comment.
🟡 MODERATE — Parentheses workaround is non-standard; prefer $\{\{ }} (Flagged by: 2/3 reviewers)
The suggested alternative if: (!github.event.repository.fork) using bare parentheses is not a documented GitHub Actions workaround. The canonical, supported fix for the YAML ! tag-indicator gotcha is if: $\{\{ !github.event.repository.fork }}. Using bare parentheses may work today via implicit expression evaluation, but it's not in GitHub's official docs and could break in future runner versions.
Suggested fix: Replace the parentheses suggestion with the canonical form, e.g.:
Either wrap in
$\{\{ }}:if: $\{\{ !github.event.repository.fork }}
| | Trigger | Why | | ||
| |---------|-----| | ||
| | `pull_request` | Causes "Approve and run" gate for ALL workflows; clicking approves everything including `pull_request_target` with full secrets. Prefer `slash_command:`, `schedule`, or `label_command:` | | ||
| | `pull_request_target` | Runs on base ref with full secrets and write token — most exploited vulnerability class. Never check out PR head SHA | |
There was a problem hiding this comment.
🟡 MODERATE — Unqualified ⛔ Avoid for pull_request contradicts Noise Reduction section (Flagged by: 3/3 reviewers after follow-up)
This table row places pull_request in ⛔ Avoid with no qualification. However, lines 170–184 of this same file actively demonstrate a pull_request + paths: pattern as a recommended noise-reduction technique, and line ~81 scopes the anti-pattern to "agentic workflows" specifically.
A developer scanning this quick-reference table could incorrectly remove a legitimate path-filtered pull_request trigger from a non-agentic CI/pre-check workflow.
Suggested fix: Add the "for agentic workflows" qualifier back to this row, e.g.:
pull_request| For agentic workflows: causes "Approve and run" gate for ALL workflows... Path-filteredpull_requestremains appropriate for deterministic CI/pre-check workflows.
| cancel-in-progress: true | ||
| ``` | ||
|
|
||
| > ⚠️ **Pre-cancellation race**: Cancellation is asynchronous — GitHub sends `SIGTERM`, waits up to 7500ms, then `SIGKILL`. Already-running steps may complete. An agent that already posted a comment cannot un-post it. A `create-pull-request` that already ran cannot un-create the PR. **Concurrency is not a substitute for idempotency.** |
There was a problem hiding this comment.
🟡 MODERATE — Cancellation signal sequence is incomplete (Flagged by: 3/3 reviewers after follow-up)
This states GitHub sends SIGTERM, waits 7500ms, then SIGKILL. According to GitHub's cancellation docs, the actual sequence is SIGINT → 7500ms → SIGTERM → 2500ms → kill process tree. SIGINT is omitted, and the timing attribution is imprecise (7500ms is the SIGINT→SIGTERM gap, not SIGTERM→SIGKILL).
Impact: Process authors who register only a SIGTERM handler would believe they have 7500ms, but would actually receive SIGINT first — potentially triggering default behavior (immediate exit) before their handler ever runs.
Suggested fix: Update to: "GitHub sends SIGINT, waits up to 7500ms, then SIGTERM, waits 2500ms, then kills the process tree."
| | `schedule` | Periodic housekeeping, polling-based PR operations | Best concurrency story; no event spamming; no approval gate | | ||
| | `labeled` / `label_command:` | Human-in-the-loop gate via label application | Triage+ required to apply label; one-shot with auto-remove | | ||
| | `issues` | Community-facing issue workflows | Immediate; `roles: all` acceptable with tight safe-outputs | | ||
| | `release` / `milestone` | Post-release/milestone automation | Trusted trigger (write+) | |
There was a problem hiding this comment.
🟡 MODERATE — roles: all "acceptable" here contradicts "NEVER" in Pattern #1 (Flagged by: 2/3 reviewers)
Pattern #1 (line ~222) says: "NEVER use all unless you've audited every safe-output". This row says roles: all is "acceptable with tight safe-outputs" — presenting it as a recommended default for issues triggers without the audit caveat.
While the intent may be consistent (both require restricted safe-outputs), the phrasing creates opposing impressions: one emphatic prohibition with conditions, one casual acceptance. A developer scanning only the quick-reference table won't see the audit requirement from Pattern #1.
Suggested fix: Align the language, e.g.: roles: all permissible after auditing all safe-outputs (see Pattern #1).
Comprehensive update to the gh-aw-guide skill incorporating validated security patterns from field research.
New content (293→410 lines, +135/-17):
Security-Critical Patterns (expanded from 4 to 5 + 3 new subsections)
merged > approved > unapproved > none > blocked), interaction matrix withroles:, configuration withtrusted-users/blocked-users/approval-labelslock-for-agent: trueguidanceissue_comment.editedfires TODAY with today's secrets on old commentslabel_command:needs triage but defaultroles:excludes itTrigger Selection Guide (new section)
All claims validated against official GitHub docs and the gh-aw integrity/triggers/concurrency references.