diff --git a/.github/agents/expert-reviewer.agent.md b/.github/agents/expert-reviewer.agent.md index 607c4474c..f12237022 100644 --- a/.github/agents/expert-reviewer.agent.md +++ b/.github/agents/expert-reviewer.agent.md @@ -13,12 +13,12 @@ You are a thorough PR reviewer for PolyPilot. Read `.github/copilot-instructions ## 1. Gather Context -``` -gh pr diff # full diff -gh pr view --json title,body # description -gh pr checks # CI status -gh pr view --json reviews,comments # existing feedback — don't duplicate -``` +Use the GitHub MCP tools (not `gh` CLI — credentials are scrubbed inside the agent container): + +- `get_pull_request` — read PR title, body, metadata +- `list_pull_request_files` — list of changed files +- `get_pull_request_diff` — full diff +- `get_pull_request_reviews` and `list_pull_request_comments` — existing feedback (don't duplicate) Read `.github/copilot-instructions.md` from the repo checkout for project conventions, architecture, and review dimensions. @@ -55,11 +55,11 @@ If a model is unavailable, proceed with the remaining models. ## 4. Post Results Before posting inline comments, validate **both** the file path AND line number: -- **Path**: must be a file that appears in `gh pr diff --name-only`. Comments on files not in the diff cause the entire review to fail with "Path could not be resolved". +- **Path**: must be a file that appears in the diff. Use `list_pull_request_files` MCP tool to get valid paths. Comments on files not in the diff cause the entire review to fail with "Path could not be resolved". - **Line**: must fall within a `@@` diff hunk for that file. Lines outside any hunk cause "Line could not be resolved". - **If either fails**: post the finding via `add_comment` as a design-level concern instead. -Run `gh pr diff --name-only` to get the list of valid paths before posting. +Use `list_pull_request_files` to get the list of valid paths before posting. 1. **Inline comments** — `create_pull_request_review_comment` for findings where BOTH path and line are valid 2. **Design-level concerns** — `add_comment` for findings outside the diff (wrong path, wrong line, or design-level). One comment, multiple bullets. @@ -67,5 +67,5 @@ Run `gh pr diff --name-only` to get the list of valid paths before post - Findings ranked by severity with consensus markers (e.g., "3/3 reviewers") - CI status, test coverage assessment, prior review status - Never mention specific model names — use "Reviewer 1/2/3" - - Always use `event: "COMMENT"` — blocking `REQUEST_CHANGES` reviews can't be auto-dismissed on re-review and cause stale blocks (see gh-aw limitation) - - **Never use APPROVE or REQUEST_CHANGES** + - `event: "COMMENT"` always — severity is communicated via emoji markers in the body, not the review event type. (Using `REQUEST_CHANGES` causes stale blocking reviews that can't be dismissed by the agent.) + - **Never use APPROVE** diff --git a/.github/instructions/gh-aw-workflows.instructions.md b/.github/instructions/gh-aw-workflows.instructions.md index ecf9307b8..bcbffa956 100644 --- a/.github/instructions/gh-aw-workflows.instructions.md +++ b/.github/instructions/gh-aw-workflows.instructions.md @@ -6,29 +6,35 @@ applyTo: # gh-aw (GitHub Agentic Workflows) Guidelines -> **Full documentation:** Read the `gh-aw-guide` skill (`.github/skills/gh-aw-guide/SKILL.md`) for complete coverage including anti-patterns table, common patterns, security boundaries, and architecture deep-dive. +When working on gh-aw workflow files, use the **`gh-aw-guide`** skill for the complete reference (architecture, security, safe outputs, fork handling, anti-patterns, troubleshooting). ## Essential Rules -1. **Never edit `.lock.yml` files** — they're auto-generated by `gh aw compile`. Always commit the lock file alongside the source `.md`. -2. **Prefer built-in gh-aw features** over manual reimplementations — check the [triggers](https://github.github.com/gh-aw/reference/triggers/), [frontmatter](https://github.github.com/gh-aw/reference/frontmatter/), and [safe-outputs](https://github.github.com/gh-aw/reference/safe-outputs/) references first. -3. **Use `slash_command:` trigger** instead of `issue_comment` + `startsWith(comment.body, '/cmd')`. -4. **Use `steps:` for GitHub API calls** — `gh` CLI credentials are scrubbed inside the agent container. -5. **Never execute untrusted PR code** (`dotnet build`, `npm install`) inside the agent — build hooks can read `COPILOT_TOKEN`. -6. **Use `Checkout-GhAwPr.ps1`** for `workflow_dispatch` triggers to check out the PR branch and restore trusted `.github/` from base. -7. **Include all PR number sources** in concurrency groups: `github.event.issue.number || github.event.pull_request.number || inputs.pr_number || github.run_id`. -8. **Block agent PR approvals** with `allowed-events: [COMMENT]` on `submit-pull-request-review` (see rule 10). -9. **Split `slash_command` + `pull_request`** into separate workflow files importing a shared `shared/*.md` — the compiler rejects them in the same file. -10. **COMMENT-only reviews** — use `allowed-events: [COMMENT]` (not `REQUEST_CHANGES`) to avoid stale blocking reviews that can't be auto-dismissed. See the "Known Limitation: Stale Blocking Reviews" section in the `gh-aw-guide` skill. +1. **Never edit `.lock.yml` files manually** — they are auto-generated by `gh aw compile` +2. **Always compile after changes**: `gh aw compile .github/workflows/.md` +3. **Always commit the lock file** alongside the source `.md` +4. **Prefer built-in gh-aw features** over manual reimplementations — invoke the `gh-aw-guide` skill for the complete anti-patterns table before implementing triggers, outputs, or scheduling manually +5. **Never execute untrusted PR code** with elevated credentials — treat PR contents as passive data only +6. **Never approve PRs from workflows** — use `allowed-events: [COMMENT]` on `submit-pull-request-review` to block approvals. Prefer `[COMMENT]` over `[COMMENT, REQUEST_CHANGES]` — `REQUEST_CHANGES` reviews from `github-actions[bot]` cannot be dismissed by the agent (no `dismiss-pull-request-review` safe output), causing stale blocking reviews (gh-aw#25439) +7. **Always use `github-token-for-extra-empty-commit:`** (PAT/App token) on `create-pull-request` — `GITHUB_TOKEN` pushes do not trigger CI +8. **Set `protected-files: fallback-to-issue`** on `create-pull-request` when the agent may touch package manifests or `.github/` — prevents PR creation from failing silently +9. **Use `Checkout-GhAwPr.ps1`** for `workflow_dispatch` workflows that check out a PR — it verifies write access and restores trusted `.github/` from base branch +10. **Set `min-integrity: approved`** on `tools.github` for workflows that process external PR content — prevents prompt injection from first-timer/contributor comments -## Quick Anti-Pattern Check +## Quick Anti-Pattern Check (Critical Subset) + +These are the most commonly missed built-in replacements. **Invoke the `gh-aw-guide` skill for the complete 21-row table** — this list covers only the highest-impact items: | If you're about to implement... | Use this built-in instead | |---------------------------------|--------------------------| | `issue_comment` + `startsWith(comment.body, '/cmd')` | `slash_command:` trigger | -| Manual emoji reaction on triggering comment | `reaction:` field under `on:` | -| Posting "workflow started/completed" status comments | `status-comment: true` under `on:` | -| Editing old comments to collapse them | `hide-older-comments: true` on `add-comment:` | -| Triggering CI on agent-created PRs | `github-token-for-extra-empty-commit:` on `create-pull-request` | +| Triggering CI on agent-created PRs manually | `github-token-for-extra-empty-commit:` on `create-pull-request` | +| No guard against agent approving PRs | `allowed-events: [COMMENT]` on `submit-pull-request-review` (avoids stale blocking reviews) | +| Manual role/bot filtering with `if:` conditions | `skip-bots:`, `skip-roles:` under `on:` | +| Manual approval gating before workflow runs | `manual-approval:` under `on:` | +| Manual label check + removal for one-shot commands | `label_command:` trigger | +| `slash_command:` without `events:` filter | `events: [pull_request_comment]` — default subscribes to ALL comment events | +| `cancel-in-progress: true` on `slash_command:` workflows | `cancel-in-progress: false` — prevents non-matching events from killing agent runs | +| Using `pull_request` trigger for agentic workflows | `slash_command:` or `schedule` — `pull_request` causes the "Approve and run" gate for ALL workflows | -See `.github/skills/gh-aw-guide/SKILL.md` for the full 19-row anti-patterns table. +For the complete anti-patterns table, architecture, security model, fork handling, and all safe output options, invoke the **`gh-aw-guide`** skill. diff --git a/.github/instructions/gh-aw-workflows.sync.yaml b/.github/instructions/gh-aw-workflows.sync.yaml index de1fb1e9b..0895f1d3a 100644 --- a/.github/instructions/gh-aw-workflows.sync.yaml +++ b/.github/instructions/gh-aw-workflows.sync.yaml @@ -1,52 +1,92 @@ -# Sync manifest for gh-aw workflow instructions -# Used by the instruction-drift skill to detect when upstream docs change. -# Run: pwsh .github/skills/instruction-drift/scripts/Check-Staleness.ps1 -SyncManifest .github/instructions/gh-aw-workflows.sync.yaml - -sync: - target: "../skills/gh-aw-guide/SKILL.md" - secondary_targets: - - "../skills/gh-aw-guide/references/architecture.md" - - # Upstream documentation pages to monitor for changes - reference_urls: - - https://github.github.com/gh-aw/reference/triggers/ - - https://github.github.com/gh-aw/reference/frontmatter/ - - https://github.github.com/gh-aw/reference/safe-outputs/ - - https://github.github.com/gh-aw/reference/command-triggers/ - - https://github.github.com/gh-aw/reference/custom-safe-outputs/ - - https://github.github.com/gh-aw/reference/triggering-ci/ - - https://github.github.com/gh-aw/patterns/monitoring/ - - https://securitylab.github.com/resources/github-actions-preventing-pwn-requests/ - - # GitHub issues whose resolution may require instruction updates - tracked_issues: - - url: https://github.com/github/gh-aw/issues/18481 - status: closed - note: "Fork support tracking — all sub-items shipped" - - url: https://github.com/github/gh-aw/issues/18518 - status: closed - note: "gh aw init now warns in forks" - - url: https://github.com/github/gh-aw/issues/18521 - status: closed - note: "Fork support docs created" - - url: https://github.com/github/gh-aw/issues/23769 - status: closed - note: "Platform auto-restores .github/ from base branch" - - url: https://github.com/github/gh-aw/issues/25439 - status: closed - note: "submit-pull-request-review allowed-events fix shipped" - # PolyPilot-specific: tracking the stale review gap - # gh-aw#25869 — another team independently adopted COMMENT-only workaround - # No upstream issue for supersede-older-reviews yet; monitor gh-aw releases - - # Sections that intentionally diverge from upstream / other repos - divergence_sections: - - "Known Limitation: Stale Blocking Reviews" - - "Security Boundaries" - - "Safe Pattern: Checkout + Restore" - - "Common Patterns" - - # Monitor for new gh-aw platform releases - releases_source: https://github.com/github/gh-aw/releases.atom - - last_reviewed: "2025-07-14" +# Drift tracking manifest for gh-aw-guide skill +# Used by the instruction-drift skill to detect when upstream sources change. +# See .github/skills/instruction-drift/SKILL.md for details. + +target: ../skills/gh-aw-guide/SKILL.md +secondary_targets: + - ../skills/gh-aw-guide/references/architecture.md + +# Sections containing our own operational knowledge (not from upstream). +# The instruction-drift skill will never suggest removing these. +divergence: + - section: "Security Boundaries" + reason: "Our defense-in-depth assessment and workflow author rules" + - section: "Safe Pattern: Checkout + Restore" + reason: "Documents our custom Checkout-GhAwPr.ps1 script" + - section: "Common Patterns" + reason: "Our specific usage patterns for pre-agent data prep, concurrency, noise reduction" + +sources: + # Reference documentation pages — check for content changes + # Core reference (high-change frequency) + - url: https://github.github.com/gh-aw/reference/triggers/ + sections: ["Anti-Patterns", "Common Patterns", "Limitations"] + - url: https://github.github.com/gh-aw/reference/frontmatter/ + sections: ["Anti-Patterns", "Architecture", "Additional Frontmatter Features"] + - url: https://github.github.com/gh-aw/reference/safe-outputs/ + sections: ["Anti-Patterns", "Safe Outputs Quick Reference"] + - url: https://github.github.com/gh-aw/reference/safe-outputs-pull-requests/ + sections: ["Safe Outputs Quick Reference"] + - url: https://github.github.com/gh-aw/reference/integrity/ + sections: ["Security Boundaries"] + coverage_gaps: + - "endorsement-reactions / disapproval-reactions (v0.68.2+)" + - "approval-labels for promoting items" + - "allowed-repos scoping" + - "integrity-proxy: false opt-out" + - "centralized management via GH_AW_GITHUB_* variables" + - "effective integrity computation order" + - "DIFC_FILTERED logging and gh aw logs --filtered-integrity" + - "public repos auto-apply min-integrity: approved when unconfigured" + - "GitHub Actions expressions for blocked-users/trusted-users/approval-labels" + - url: https://github.github.com/gh-aw/reference/custom-safe-outputs/ + sections: ["Anti-Patterns", "Safe Outputs Quick Reference"] + - url: https://github.github.com/gh-aw/reference/triggering-ci/ + sections: ["Limitations", "Troubleshooting"] + - url: https://github.github.com/gh-aw/reference/command-triggers/ + sections: ["Anti-Patterns"] + + # Extended reference (lower change frequency, but contain important context) + - url: https://github.github.com/gh-aw/reference/engines/ + sections: ["Additional Frontmatter Features"] + - url: https://github.github.com/gh-aw/reference/network/ + sections: ["Security Boundaries"] + - url: https://github.github.com/gh-aw/reference/github-tools/ + sections: ["Security Boundaries"] + - url: https://github.github.com/gh-aw/reference/imports/ + sections: ["Common Patterns"] + - url: https://github.github.com/gh-aw/reference/workflow-structure/ + sections: ["Architecture"] + + # Patterns (best practices that inform our anti-patterns table) + - url: https://github.github.com/gh-aw/patterns/trial-ops/ + sections: ["Common Patterns"] + - url: https://github.github.com/gh-aw/patterns/chat-ops/ + sections: ["Common Patterns", "Anti-Patterns"] + - url: https://github.github.com/gh-aw/patterns/label-ops/ + sections: ["Common Patterns"] + + # Tracked GitHub issues — check open/closed state + # NOTE: Only set resolution_expected: true for issues whose closure + # has NOT yet been incorporated into our docs. All 5 below are already + # Tracked upstream issues — all resolved. resolution_expected: true means + # the script expects them closed and won't flag them as stale. + - issue: github/gh-aw#18481 + resolution_expected: true + - issue: github/gh-aw#18518 + resolution_expected: true + - issue: github/gh-aw#18521 + resolution_expected: true + - issue: github/gh-aw#23769 + resolution_expected: true + - issue: github/gh-aw#25439 + resolution_expected: true + + # GitHub releases — check for new versions + - releases: github/gh-aw + +style: | + Match existing section structure. Use tables for feature comparisons. + Include code examples for common patterns. Mark items as anti-patterns + when a manual reimplementation should use a built-in feature instead. + Keep security guidance precise — never simplify away nuance. diff --git a/.github/skills/gh-aw-guide/SKILL.md b/.github/skills/gh-aw-guide/SKILL.md index f5656de9a..fa793959f 100644 --- a/.github/skills/gh-aw-guide/SKILL.md +++ b/.github/skills/gh-aw-guide/SKILL.md @@ -1,15 +1,13 @@ --- name: gh-aw-guide description: >- - Comprehensive guide for building and maintaining GitHub Agentic Workflows - (gh-aw). Covers architecture, security boundaries, fork handling, safe - outputs, anti-patterns, compilation, and troubleshooting. Use when creating - or editing gh-aw workflow .md files, writing safe-outputs configurations, - configuring fork PR handling, setting up integrity filtering, debugging - "why doesn't my workflow trigger", or any task involving - .github/workflows/*.md or .lock.yml files. Also use when asked about gh-aw - features, slash commands, pre-agent-steps, protected files, or agentic - workflow security. + Comprehensive guide for building and maintaining GitHub Agentic Workflows (gh-aw). + Covers architecture, security boundaries, fork handling, safe outputs, anti-patterns, + compilation, and troubleshooting. Use when creating or editing gh-aw workflow .md files, + writing safe-outputs configurations, configuring fork PR handling, setting up integrity + filtering, debugging "why doesn't my workflow trigger", or any task involving + .github/workflows/*.md or .lock.yml files. Also use when asked about gh-aw features, + slash commands, pre-agent-steps, protected files, or agentic workflow security. --- # gh-aw (GitHub Agentic Workflows) Guide @@ -61,7 +59,10 @@ gh aw compile .github/workflows/.md | Custom post-processing jobs for agent output | `safe-outputs.jobs:` custom jobs with MCP tool access | [Custom Safe Outputs](https://github.github.com/gh-aw/reference/custom-safe-outputs/) | | Wrapping GitHub Actions as agent-callable tools | `safe-outputs.actions:` action wrappers | [Custom Safe Outputs](https://github.github.com/gh-aw/reference/custom-safe-outputs/) | | Triggering CI on agent-created PRs | `github-token-for-extra-empty-commit:` on `create-pull-request` | [Triggering CI](https://github.github.com/gh-aw/reference/triggering-ci/) | -| No guard against agent approving/blocking PRs | `allowed-events: [COMMENT]` on `submit-pull-request-review` — blocks both APPROVE and stale REQUEST_CHANGES | [Safe Outputs](https://github.github.com/gh-aw/reference/safe-outputs/) | +| No guard against agent approving PRs | `allowed-events: [COMMENT]` on `submit-pull-request-review` (prefer over `[COMMENT, REQUEST_CHANGES]` to avoid stale blocking reviews) | [Safe Outputs](https://github.github.com/gh-aw/reference/safe-outputs/) | +| `slash_command:` without `events:` filter (subscribes to ALL comment events) | `events: [pull_request_comment]` or `events: [issue_comment]` | [Command Triggers](https://github.github.com/gh-aw/reference/command-triggers/) | +| `cancel-in-progress: true` on `slash_command:` workflows | `cancel-in-progress: false` — non-matching events cancel in-progress agent runs | [Concurrency](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/using-concurrency) | +| Using `pull_request` trigger for agentic workflows | `slash_command:` or `schedule` — `pull_request` causes the "Approve and run" gate for ALL workflows | [Triggers](https://github.github.com/gh-aw/reference/triggers/) | **Note:** gh-aw is actively developed. If a capability feels like something a framework would provide natively, check the reference docs — it probably exists even if it's not in this table yet. @@ -95,14 +96,57 @@ safe-outputs: ### Concurrency -Include all trigger-specific PR number sources: +Include all trigger-specific PR number sources. **Use `cancel-in-progress: false` for `slash_command:` workflows** — a non-matching event (ordinary comment) in the same concurrency group can cancel an in-progress matching run (the actual `/command`), killing the agent mid-execution: ```yaml +# For slash_command workflows — never cancel in-progress concurrency: group: "my-workflow-${{ github.event.issue.number || github.event.pull_request.number || inputs.pr_number || github.run_id }}" + cancel-in-progress: false + +# For schedule/workflow_dispatch only — safe to cancel +concurrency: + group: "my-workflow-${{ github.ref || github.run_id }}" cancel-in-progress: true ``` +### `slash_command:` Event Subscription + +`slash_command:` compiles to broad event subscriptions — by default it listens to **all** comment-related events (issue open/edit, PR open/edit, every comment, every review comment, every discussion comment), then filters post-activation. This means: + +- **Runner cost**: The pre-activation job runs on every matching event (~5-30s each), even when skipped. On busy repos this can be hundreds of skipped runs per day. +- **Actions UI noise**: Operators learn to ignore "skipped" runs and may miss real failures. +- **Concurrency collisions**: Non-matching events in the same concurrency group can cancel matching ones (see above). + +**Always narrow `events:`** to the minimum needed: + +```yaml +on: + slash_command: + name: review + events: [pull_request_comment] # Only PR comments, not issues/discussions +``` + +### The "Approve and Run Workflows" Gate + +The `pull_request` trigger causes an "Approve and run workflows" button for first-time fork contributors. **This gate is dangerous, not protective**: + +1. **Alert fatigue** — After clicking through dozens of legitimate first-time PRs, the click becomes muscle memory +2. **No per-workflow granularity** — A single click approves ALL gated workflows, including any `pull_request_target` workflows with full secrets +3. **No diff preview** — The UI shows no preview of what will execute or which secrets are exposed + +**Design rule**: Assume the approval gate will always be clicked. The only safe workflows are ones that produce the same outcome whether the actor is trusted or untrusted. Prefer `issue_comment`/`slash_command:` (not subject to the gate) or `schedule`/`workflow_dispatch` over `pull_request` when possible. + +### LabelOps + +gh-aw provides label-based triggering patterns for both one-shot commands and persistent state tracking: + +- **`label_command:`** — One-shot command triggered by applying a label. The label is auto-removed after the workflow fires, making it self-resetting. Use for operations like "apply this label to trigger a review". +- **`names:` filtering** — Filter label events to specific label names for persistent label-state awareness. +- **`remove_label: false`** — Keep the label after triggering (for persistent state markers rather than one-shot commands). + +See the [LabelOps pattern guide](https://github.github.com/gh-aw/patterns/label-ops/) for detailed examples and best practices. + ### Noise Reduction Filter `pull_request` triggers to relevant paths and add a gate step: @@ -142,12 +186,12 @@ For `pull_request` + fork support (not `workflow_dispatch`): add `forks: ["*"]` These four patterns are the most commonly missed when building secure workflows. Use all where applicable: -**1. Prevent accidental PR approvals** — always restrict review workflows; otherwise the agent can approve PRs and bypass branch protection rules (gh-aw#25439): +**1. Prevent accidental PR approvals** — always restrict review workflows; otherwise the agent can approve PRs and bypass branch protection rules (gh-aw#25439). Use `[COMMENT]` to avoid stale blocking reviews that can't be dismissed: ```yaml safe-outputs: submit-pull-request-review: - allowed-events: [COMMENT] # Blocks APPROVE and REQUEST_CHANGES — stale blocking reviews can't be auto-dismissed + allowed-events: [COMMENT] # Blocks APPROVE; avoids un-dismissable REQUEST_CHANGES ``` **2. CI triggering + protected file safety** for agent-created PRs — `GITHUB_TOKEN` pushes don't trigger CI; a PAT/App token is required. `protected-files` controls what happens when the agent modifies package manifests or `.github/`: @@ -156,7 +200,7 @@ safe-outputs: safe-outputs: create-pull-request: github-token-for-extra-empty-commit: ${{ secrets.PAT_OR_APP_TOKEN }} # Required to trigger CI - protected-files: fallback-to-issue # Create issue instead of failing if agent touches .github/ or package manifests + protected-files: fallback-to-issue # Create issue instead of failing if agent touches .github/ or package manifests # protected-files: blocked (default) | allowed (disables protection) ``` @@ -165,7 +209,7 @@ safe-outputs: ```yaml tools: github: - min-integrity: approved # Filters FIRST_TIMER / CONTRIBUTOR content; use on workflows that process external PR content + min-integrity: approved # Filters FIRST_TIMER / CONTRIBUTOR content; use on workflows that process external PR content ``` **4. Fork PR checkout for `workflow_dispatch`** — the platform's `checkout_pr_branch.cjs` is skipped for `workflow_dispatch`, so you **must** use `.github/scripts/Checkout-GhAwPr.ps1` to check out the PR branch, verify write access, reject fork PRs, and restore trusted `.github/` from the base branch. Without it, the agent evaluates the workflow branch instead of the PR: @@ -188,29 +232,32 @@ resources: # Companion fil - triage-issue.md - shared/helper-action.yml labels: ["automation", "ci"] # For gh aw status --label filtering +checkout: false # Skip repo checkout (for workflows that only use MCP/API, no source needed) + +rate-limit: # Throttle slash commands to prevent abuse + max: 5 # Max invocations per window + window: 60 # Window in seconds -runtimes: # Override default runtime versions +runtimes: # Override default runtime versions dotnet: version: "9.0" node: version: "22" -imports: # APM package dependencies +imports: # APM package dependencies - uses: shared/apm.md with: packages: - microsoft/apm-sample-package ``` -Supported runtimes: `node`, `python`, `go`, `uv`, `bun`, `deno`, `ruby`, `java`, `dotnet`, `elixir`. +**`checkout: false`** — Skip the default repository checkout when the workflow doesn't need source code (e.g., ChatOps commands that only call APIs via `web-fetch`). Saves ~10-30s of runner time. -## Known Limitation: Stale Blocking Reviews +**`rate-limit:`** — Throttle slash command invocations to prevent abuse or accidental spam. The `max` field limits invocations per `window` seconds. Useful for commands that call external APIs or create issues. -`submit-pull-request-review` with `REQUEST_CHANGES` creates a blocking review that persists even after all findings are fixed and a re-review runs. gh-aw has no `dismiss-pull-request-review` safe output and forbids `pull-requests: write`, so stale bot reviews cannot be auto-dismissed. The `add-comment` output has `hide-older-comments: true` for this lifecycle, but reviews have no equivalent. +**Available tools:** `web-fetch` (fetch URLs), `bash` (shell commands), GitHub MCP toolsets (`pull_requests`, `repos`, `issues`, etc.). Use `tools: [web-fetch]` for workflows that call external APIs. -**Our workaround:** Use `allowed-events: [COMMENT]` only — reviews communicate severity via 🔴/🟡/🟢 in the body but never block merging. This loses the GitHub-native "Changes requested" badge and merge-blocking semantics. - -**Upstream request:** A `supersede-older-reviews: true` option on `submit-pull-request-review` would solve this — auto-dismiss previous bot reviews from the same workflow when posting a new one, analogous to `hide-older-comments: true`. See gh-aw#25869 for another team that independently adopted the same COMMENT-only workaround. +Supported runtimes: `node`, `python`, `go`, `uv`, `bun`, `deno`, `ruby`, `java`, `dotnet`, `elixir`. ## When to Read the Full Reference diff --git a/.github/skills/gh-aw-guide/references/architecture.md b/.github/skills/gh-aw-guide/references/architecture.md index 763103aac..5cba02719 100644 --- a/.github/skills/gh-aw-guide/references/architecture.md +++ b/.github/skills/gh-aw-guide/references/architecture.md @@ -1,6 +1,6 @@ -# gh-aw Architecture Deep-Dive +# gh-aw Architecture & Security Reference -This document provides the full architecture, security model, and reference details for GitHub Agentic Workflows (gh-aw). Read the parent `SKILL.md` for the quick-start guide and common patterns. +Deep reference for gh-aw execution model, security boundaries, fork handling, safe outputs, and troubleshooting. Read this file when the SKILL.md quick-start and common patterns aren't sufficient. ## Execution Model @@ -27,9 +27,9 @@ agent job: | Agent container | ❌ Scrubbed | ❌ Scrubbed | ❌ Scrubbed | ✅ But sandboxed | | `post-steps:` | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes — runs after agent | -**⚠️ Agent container credential nuance:** `GITHUB_TOKEN` and `gh` CLI credentials are scrubbed inside the agent container. However, `COPILOT_TOKEN` (used for LLM inference) is present in the environment via `--env-all`. Any subprocess (e.g., `dotnet build`, `npm install`) inherits this variable. The AWF network firewall, `redact_secrets.cjs` (post-agent log scrubbing), and the threat detection agent limit the blast radius. See [Security Boundaries](#security-boundaries) below. +**Agent container credential nuance:** `GITHUB_TOKEN` and `gh` CLI credentials are scrubbed inside the agent container. However, `COPILOT_TOKEN` (used for LLM inference) is present in the environment via `--env-all`. Any subprocess (e.g., `dotnet build`, `npm install`) inherits this variable. The AWF network firewall, `redact_secrets.cjs` (post-agent log scrubbing), and the threat detection agent limit the blast radius. -## Step Ordering +### Step Ordering User `steps:` run in the **pre-activation job** (before the agent job starts). Within the agent job, the ordering is: platform steps → `pre-agent-steps:` → agent → `post-steps:`. @@ -46,19 +46,21 @@ pre-agent-steps: **`post-steps:`** run after the agent completes but before safe-outputs. Use these for cleanup, metrics, or post-processing. -## Prompt Rendering +### Prompt Rendering The prompt is built in the **activation job** via `{{#runtime-import .github/workflows/.md}}`. This reads the `.md` file from the **base branch** workspace (before any PR checkout). The rendered prompt is uploaded as an artifact and downloaded by the agent job. - The agent prompt is always the base branch version — fork PRs cannot alter it - The prompt references files on disk (e.g., `SKILL.md`) — those files must exist in the agent's workspace -## Fork PR Activation Gate +### Fork PR Activation Gate By default, `gh aw compile` automatically injects a fork guard into the activation job's `if:` condition: `head.repo.id == repository_id`. This blocks fork PRs on `pull_request` events. To **allow fork PRs**, add `forks: ["*"]` to the `pull_request` trigger in the `.md` frontmatter. The compiler removes the auto-injected guard from the compiled `if:` conditions. This is safe when the workflow uses the `Checkout-GhAwPr.ps1` pattern (checkout + trusted-infra restore) and the agent is sandboxed. +--- + ## Security Boundaries ### Key Principles (from [GitHub Security Lab](https://securitylab.github.com/resources/github-actions-preventing-pwn-requests/)) @@ -129,9 +131,41 @@ Configure behavior with `protected-files:` on the safe output: - ✅ **DO** treat PR contents as passive data (read, analyze, diff) - ✅ **DO** run data-gathering scripts in `steps:` (pre-agent, trusted context) not inside the agent - ✅ **DO** use `Checkout-GhAwPr.ps1` for `workflow_dispatch` to restore trusted `.github/` from base +- ✅ **DO** narrow `slash_command: events:` to the minimum needed (e.g., `[pull_request_comment]`) +- ✅ **DO** use `cancel-in-progress: false` for `slash_command:` workflows to prevent non-matching events from killing in-progress agent runs +- ✅ **DO** prefer `slash_command:` or `schedule` over `pull_request` trigger — `pull_request` causes the "Approve and run" gate that approves ALL workflows with a single click - ❌ **DO NOT** run `dotnet build`, `npm install`, or any build command on untrusted PR code inside the agent — build tool hooks (MSBuild targets, postinstall scripts) can read `COPILOT_TOKEN` from the environment - ❌ **DO NOT** execute workspace scripts (`.ps1`, `.sh`, `.py`) after checking out a fork PR in `steps:` — those run with `GITHUB_TOKEN` -- ❌ **DO NOT** set `roles: all` on workflows that process PR content — this allows any user to trigger the workflow +- ❌ **DO NOT** set `roles: all` on workflows that process PR content — the agent's `permissions:` and `safe-outputs:` determine what actions are taken, NOT the actor's role. Setting `roles: all` gives any read-only user bot-level write access to anything the workflow grants. + +### Authorization Model (`on.roles:`) + +**What the agent can do is determined by the workflow's `permissions:` and `safe-outputs:` — NOT by the actor who fired it.** When a workflow accepts a read-only contributor as the trigger (via `roles: all`), that contributor effectively gets bot-level write access to anything the workflow grants. + +`on.roles:` defaults to `[admin, maintainer, write]`. This deny-by-default gate prevents read-only users from inducing the bot to act with elevated permissions. Available roles: `admin`, `maintainer`/`maintain`, `write`, `triage`, `read`, `all`. + +**Key interactions:** +- A `read` user can fire any `slash_command:`, `issues`, `issue_comment`, or `discussion` trigger — they just can't pass the default `roles:` check +- A `read` user **cannot** fire `label_command:` (requires `triage` to apply label) or `workflow_dispatch` (requires `write`) +- `triage` users can apply labels but are excluded from the default `roles:` allowlist — `label_command:` workflows need `roles: [admin, maintainer, write, triage]` to work for triagers + +### Concurrency Race Conditions + +With `cancel-in-progress: true` on `slash_command:` workflows, a **non-matching event** (e.g., an ordinary comment) in the same concurrency group can cancel an **in-progress matching run** (the actual `/command`). This happens because `slash_command:` compiles to broad `issue_comment` event subscriptions — every comment triggers the pre-activation job, which runs in the same concurrency group. + +**Fix:** Always use `cancel-in-progress: false` for `slash_command:` workflows. Redundant runs from rapid re-invocation are preferable to killed agents. + +### The "Approve and Run" Gate + +The `pull_request` trigger causes an "Approve and run workflows" button for first-time fork contributors. **This gate is dangerous, not protective:** + +1. **Alert fatigue** — After clicking through dozens of legitimate first-time PRs, the click becomes muscle memory +2. **No per-workflow granularity** — A single click approves ALL gated workflows, including any `pull_request_target` workflows with full secrets +3. **No diff preview** — The UI shows no preview of what will execute or which secrets are exposed + +**Design rule**: Assume the approval gate will always be clicked. Prefer `slash_command:` or `schedule` over `pull_request` when possible. + +--- ## Fork PR Handling @@ -145,6 +179,59 @@ Reference: https://securitylab.github.com/resources/github-actions-preventing-pw The platform now **automatically preserves `.github/` and `.agents/` from the base branch**. The activation job saves these directories as an artifact, and after `checkout_pr_branch.cjs` checks out the PR branch, the platform restores them from the artifact. Additionally, `.mcp.json` is deleted from the workspace to prevent injection. This means fork PRs can no longer overwrite agent infrastructure (skills, instructions, copilot-instructions) by including modified copies in their branch. +**What #23769 fixed (and what it didn't):** + +| Before #23769 | After #23769 | +|---------------|-------------| +| Fork PRs inject modified `.github/skills/`, `.github/instructions/`, `.agents/` | ✅ Platform restores these from base branch artifact | +| Fork PRs inject `.mcp.json` to add malicious MCP servers | ✅ Platform deletes `.mcp.json` after checkout | +| User `steps:` run BEFORE `checkout_pr_branch.cjs` — restore was overwritten | ✅ Platform restore happens AFTER checkout | +| `pre-agent-steps:` not available | ✅ `pre-agent-steps:` run after restore, before agent | + +**Remaining risks (not fixed by #23769):** +- `steps:` and `pre-agent-steps:` that execute workspace code after checkout still run with `GITHUB_TOKEN` — if they run fork PR scripts, it's a pwn-request +- The agent container has `COPILOT_TOKEN` in the environment — build commands (`dotnet build`, `npm install`) executed by the agent on fork PR code can read it via build hooks +- `workflow_dispatch` skips `checkout_pr_branch.cjs` entirely — use `Checkout-GhAwPr.ps1` for defense-in-depth + +### Dangerous Triggers Checklist + +Use this checklist when reviewing any workflow that uses high-risk triggers. The platform's #23769 restore makes these **safer** but not **safe** — the remaining risks require workflow-author discipline. + +#### `pull_request_target` +- ⚠️ Grants **write permissions and secrets access** even for fork PRs +- ✅ `.github/` is now restored from base branch (gh-aw#23769) +- ❌ `steps:` and `pre-agent-steps:` still run fork code with `GITHUB_TOKEN` +- **Checks:** + - [ ] No `steps:` or `pre-agent-steps:` execute workspace scripts after checkout + - [ ] No build commands in the agent prompt (agent has `COPILOT_TOKEN`) + - [ ] `roles:` is NOT `all` (gate on write-access minimum) + - [ ] `min-integrity: approved` is set if processing PR content + - [ ] `protected-files:` is set if `create-pull-request` or `push-to-pull-request-branch` is used + +#### `workflow_run` +- ⚠️ Inherits secrets access, runs after another workflow completes +- ✅ Separates privilege from code execution (the pattern gh-aw uses internally) +- ❌ Artifacts from the first run could contain executable payloads +- **Checks:** + - [ ] `branches:` is restricted (not open to all branches) + - [ ] Artifact contents are treated as untrusted data (never `eval`, `source`, or execute) + - [ ] The triggering workflow is pinned (not modifiable by fork PRs) + +#### `push` with broad branch patterns +- ⚠️ Runs with write permissions on every push +- **Checks:** + - [ ] `branches:` is narrowed to specific branches (e.g., `[main]`), never `['**']` + - [ ] No `roles: all` — meaningless on push but indicates careless authoring + +#### `issue_comment` / `slash_command` on fork PRs +- ✅ Platform restores `.github/` from base branch (gh-aw#23769) +- ✅ Only write-access users can trigger (default `roles:`) +- ⚠️ The PR code is checked out — agent reads it as passive data (safe), but `steps:` must not execute it +- **Checks:** + - [ ] `events:` is narrowed (e.g., `[pull_request_comment]` not all events) + - [ ] `cancel-in-progress: false` (prevent non-matching comments from killing agent) + - [ ] No workspace script execution in `steps:` after checkout + ### Fork PR Behavior by Trigger | Trigger | `checkout_pr_branch.cjs` runs? | Fork handling | @@ -210,6 +297,8 @@ If you need to run scripts, either: 1. Run them **before** the checkout (from the base branch) 2. Run them **inside the agent container** (sandboxed, no tokens) +--- + ## Safe Outputs Quick Reference Safe outputs enforce security through separation: agents run read-only and request actions via structured output, while separate permission-controlled jobs execute those requests. @@ -226,7 +315,7 @@ Safe outputs enforce security through separation: agents run read-only and reque | **System (auto-enabled)** | `noop`, `missing-tool`, `missing-data` | | **Custom** | `jobs:` (custom post-processing with MCP tool access), `actions:` (GitHub Action wrappers) | -### Key Safe Output Features for Our Workflows +### Key Safe Output Features **`create-pull-request` notable options:** - `draft: true` — Enforced as policy (agent cannot override) @@ -236,23 +325,31 @@ Safe outputs enforce security through separation: agents run read-only and reque - `protected-files: fallback-to-issue` — Create issue instead of failing when protected files modified - `base-branch: "vnext"` — Target non-default branch - `auto-close-issue: false` — Don't add `Fixes #N` to PR description +- `allowed-events: [COMMENT]` — On `submit-pull-request-review`, blocks agent from approving PRs (bypasses branch protection). **Always use this** for review workflows. +- **Stale review limitation**: Prefer `allowed-events: [COMMENT]` unless you need the "Changes requested" badge. `REQUEST_CHANGES` reviews from `github-actions[bot]` cannot be dismissed by the agent (no `dismiss-pull-request-review` safe output, `pull-requests: write` rejected by compiler). A stale blocking review persists until a human dismisses it manually. **`add-comment` notable options:** - `hide-older-comments: true` — Collapse previous comments from same workflow - `max: N` — Limit comments per run (default: 1) - `target: "*"` — Required for `workflow_dispatch` (no triggering PR context) +--- + ## Limitations | What | Behavior | Workaround | |------|----------|------------| | Agent-created PRs don't trigger CI | GitHub Actions ignores pushes from `GITHUB_TOKEN` | Use `github-token-for-extra-empty-commit:` with a PAT or GitHub App token on `create-pull-request`. See [Triggering CI](https://github.github.com/gh-aw/reference/triggering-ci/) | +| Stale `REQUEST_CHANGES` reviews | Agent reviews with `REQUEST_CHANGES` block PRs and can't be dismissed (no `dismiss-pull-request-review` safe output) | Use `allowed-events: [COMMENT]` — communicate severity via markers in the review body instead | +| `slash_command:` runs on every comment event | Default subscription listens to all comment-related events, not just the `/command` | Always narrow `events:` (e.g., `events: [pull_request_comment]`). Each skipped run costs ~5-30s of runner time. | +| Non-matching event cancels matching run | With `cancel-in-progress: true`, a non-matching comment in the same concurrency group cancels an in-progress `/command` run | Use `cancel-in-progress: false` for `slash_command:` workflows | +| `pull_request` trigger causes "Approve and run" gate | The button approves ALL gated workflows with a single click — no per-workflow granularity, no diff preview | Prefer `slash_command:` or `schedule` over `pull_request`; assume the gate will always be clicked | | `--allow-all-tools` in lock.yml | Emitted by `gh aw compile` | Cannot override from `.md` source | | `gh` CLI inside agent | Credentials scrubbed | Use `steps:` for API calls, or MCP tools | | `issue_comment` trigger | Requires workflow on default branch | Must merge to `main` before `/slash-commands` work | | Duplicate runs | gh-aw sometimes creates 2 runs per dispatch | Harmless, use concurrency groups | -| `slash_command` + `pull_request` in same workflow | Compiler rejects: `cannot use 'slash_command' with 'pull_request' in the same workflow` | Create two separate workflow files importing the same `shared/*.md`. E.g., `review.agent.md` (slash_command) + `review-on-open.agent.md` (pull_request). Both import `shared/review-shared.md` for shared config and orchestration instructions. | -| Stale `CHANGES_REQUESTED` bot reviews | `REQUEST_CHANGES` creates blocking review that persists after re-review | Use `allowed-events: [COMMENT]` to avoid blocking reviews; manually dismiss stale reviews via API | + +--- ## Upstream References (All Resolved) @@ -264,7 +361,9 @@ These issues are now **all closed** — documented here for historical context: | [gh-aw#18518](https://github.com/github/gh-aw/issues/18518) | ✅ Closed | `gh aw init` now warns in forks, lists required secrets | | [gh-aw#18521](https://github.com/github/gh-aw/issues/18521) | ✅ Closed | Fork support docs created — forks are not supported by default; agents will not run on fork PRs unless `forks:` is configured | | [gh-aw#23769](https://github.com/github/gh-aw/issues/23769) | ✅ Closed | Platform now auto-restores `.github/` and `.agents/` from base branch after checkout; `.mcp.json` deleted to prevent injection | -| [gh-aw#25439](https://github.com/github/gh-aw/issues/25439) | ✅ Closed | `submit-pull-request-review` safe output previously allowed agents to accidentally approve PRs, bypassing branch protection. Upstream resolution: `allowed-events` filtering. PolyPilot uses `[COMMENT]` only — see Known Limitation: Stale Blocking Reviews | +| [gh-aw#25439](https://github.com/github/gh-aw/issues/25439) | ✅ Closed | `submit-pull-request-review` safe output previously allowed agents to accidentally approve PRs, bypassing branch protection. Resolution: use `allowed-events: [COMMENT, REQUEST_CHANGES]` to block approvals at infrastructure level | + +--- ## Troubleshooting @@ -279,4 +378,8 @@ These issues are now **all closed** — documented here for historical context: | `/slash-command` doesn't trigger | Workflow not on default branch | Merge to `main` first | | Agent sees stale issue/PR content | Integrity filtering removed it | Check `min-integrity` level; content from `FIRST_TIMER` is filtered at `approved` | | Protected file error on PR creation | Agent modified `.github/` or package manifests | Set `protected-files: fallback-to-issue` or `allowed` if intentional | -| Stale `CHANGES_REQUESTED` review blocks PR | Bot review from earlier run persists after fixes | Use `allowed-events: [COMMENT]` to avoid blocking reviews; manually dismiss stale reviews via API | +| Stale blocking review after fixes | Agent posted `REQUEST_CHANGES` but can't dismiss it | Switch to `allowed-events: [COMMENT]`; use severity markers in body instead | +| Agent run killed by unrelated comment | `cancel-in-progress: true` + `slash_command:` with broad event subscription | Use `cancel-in-progress: false`; narrow `events:` to reduce concurrency group collisions | +| Hundreds of "skipped" runs per day | `slash_command:` default subscribes to ALL comment events | Narrow `events:` field; accept remaining noise as cost of low-latency invocation | +| `label_command:` denies triage users | `triage` role not in default `on.roles:` allowlist | Add `roles: [admin, maintainer, write, triage]` | +| Forked workflows fire unexpectedly in forks | Workflows copied to fork run on routine events inside the fork | Add job-level guard: `if: github.event.repository.fork == false \|\| github.event_name == 'workflow_dispatch'` | diff --git a/.github/skills/gh-aw-guide/scripts/Check-WorkflowSecurity.ps1 b/.github/skills/gh-aw-guide/scripts/Check-WorkflowSecurity.ps1 new file mode 100644 index 000000000..4924fc68b --- /dev/null +++ b/.github/skills/gh-aw-guide/scripts/Check-WorkflowSecurity.ps1 @@ -0,0 +1,225 @@ +<# +.SYNOPSIS + Scans gh-aw workflow .md files for dangerous trigger patterns and + missing security gates. + +.DESCRIPTION + Checks each workflow source file (.github/workflows/*.md, excluding + lock files and shared/) for: + - pull_request_target without min-integrity or role restrictions + - workflow_run without branch restrictions + - push with overly broad branch patterns + - roles: all on workflows that process PR content + - slash_command with cancel-in-progress: true (agent-kill risk) + - Missing allowed-events on submit-pull-request-review + - steps: that execute workspace scripts after checkout + - Missing protected-files on create-pull-request / push-to-pull-request-branch + + Outputs findings as a JSON array. Exit 0 = clean, exit 1 = findings. + +.PARAMETER WorkflowDir + Directory containing workflow .md files. Default: .github/workflows + +.EXAMPLE + pwsh Check-WorkflowSecurity.ps1 + pwsh Check-WorkflowSecurity.ps1 -WorkflowDir .github/workflows +#> + +[CmdletBinding()] +param( + [string]$WorkflowDir = ".github/workflows" +) + +$ErrorActionPreference = 'Stop' + +function Get-WorkflowFiles { + Get-ChildItem -Path $WorkflowDir -Filter '*.md' -File | + Where-Object { $_.Name -notmatch '\.lock\.' -and $_.Directory.Name -ne 'shared' } +} + +function Test-Workflow { + param( + [string]$Path, + [string]$Content + ) + + $findings = @() + $name = Split-Path -Leaf $Path + + # Extract YAML frontmatter + $frontmatter = "" + if ($Content -match '(?s)^---\s*\n(.*?)\n---') { + $frontmatter = $Matches[1] + } + + # --- Trigger checks --- + + # pull_request_target without safety gates + if ($frontmatter -match 'pull_request_target') { + if ($frontmatter -notmatch 'min-integrity') { + $findings += @{ + file = $name + severity = "HIGH" + rule = "pull_request_target-no-integrity" + message = "pull_request_target trigger without min-integrity filtering. Fork PR content is unfiltered — prompt injection risk." + fix = "Add tools.github.min-integrity: approved" + } + } + if ($frontmatter -match 'roles:\s*all') { + $findings += @{ + file = $name + severity = "CRITICAL" + rule = "pull_request_target-roles-all" + message = "pull_request_target with roles: all — any user can trigger a workflow with write permissions and secrets access." + fix = "Remove roles: all or restrict to [admin, maintainer, write]" + } + } + } + + # workflow_run without branch restrictions + if ($frontmatter -match 'workflow_run') { + if ($frontmatter -notmatch 'branches:') { + $findings += @{ + file = $name + severity = "MODERATE" + rule = "workflow_run-no-branches" + message = "workflow_run trigger without branch restrictions. Any branch can trigger this privileged workflow." + fix = "Add branches: [main] under workflow_run" + } + } + } + + # push with broad patterns + if ($frontmatter -match 'push:') { + if ($frontmatter -match 'branches:\s*\[\s*[''"]?\*\*[''"]?\s*\]') { + $findings += @{ + file = $name + severity = "HIGH" + rule = "push-wildcard-branches" + message = "push trigger with branches: ['**'] — runs on every push to every branch with write permissions." + fix = "Narrow to specific branches: [main] or [main, 'release/*']" + } + } + } + + # roles: all on PR-processing workflows + if ($frontmatter -match 'roles:\s*all' -and $frontmatter -match '(pull_request|issue_comment|slash_command)') { + $findings += @{ + file = $name + severity = "HIGH" + rule = "roles-all-pr-processing" + message = "roles: all on a workflow that processes PR content. Any user gets bot-level write access to whatever safe-outputs grant." + fix = "Restrict to roles: [admin, maintainer, write]" + } + } + + # --- Concurrency checks --- + + # slash_command with cancel-in-progress: true + if ($frontmatter -match 'slash_command' -and $frontmatter -match 'cancel-in-progress:\s*true') { + $findings += @{ + file = $name + severity = "MODERATE" + rule = "slash-command-cancel-in-progress" + message = "slash_command with cancel-in-progress: true. Non-matching comments can kill in-progress agent runs." + fix = "Use cancel-in-progress: false for slash_command workflows" + } + } + + # --- Safe output checks --- + + # submit-pull-request-review without allowed-events + if ($frontmatter -match 'submit-pull-request-review' -and $frontmatter -notmatch 'allowed-events') { + $findings += @{ + file = $name + severity = "HIGH" + rule = "review-no-allowed-events" + message = "submit-pull-request-review without allowed-events restriction. Agent can APPROVE PRs, bypassing branch protection." + fix = "Add allowed-events: [COMMENT] (or [COMMENT, REQUEST_CHANGES] if dismissal is acceptable)" + } + } + + # create-pull-request or push-to-pull-request-branch without protected-files + if ($frontmatter -match '(create-pull-request|push-to-pull-request-branch)' -and $frontmatter -notmatch 'protected-files') { + $findings += @{ + file = $name + severity = "LOW" + rule = "code-push-no-protected-files" + message = "Code push safe-output without explicit protected-files policy. Default is 'blocked' which is safe but may cause unexpected failures." + fix = "Add protected-files: fallback-to-issue (or blocked/allowed explicitly)" + } + } + + # --- Execution safety checks --- + + # Check for script execution patterns in steps/pre-agent-steps + $stepsSection = "" + if ($frontmatter -match '(?s)((?:pre-agent-)?steps:\s*\n(?:\s+-.*\n?)+)') { + $stepsSection = $Matches[1] + } + + if ($stepsSection -match '(run:|pwsh|bash|python|node)\s.*\.(ps1|sh|py|js)') { + if ($frontmatter -match '(pull_request_target|pull_request|issue_comment|slash_command)') { + $findings += @{ + file = $name + severity = "HIGH" + rule = "steps-execute-workspace-scripts" + message = "steps: or pre-agent-steps: execute workspace scripts (.ps1/.sh/.py/.js) after PR checkout. Fork PRs can inject malicious scripts that run with GITHUB_TOKEN." + fix = "Move script execution to before checkout, or only run scripts from the base branch (use Checkout-GhAwPr.ps1 pattern)" + } + } + } + + return $findings +} + +# --- Main --- + +Write-Host "🔒 Workflow Security Scanner" -ForegroundColor Cyan +Write-Host "Directory: $WorkflowDir" +Write-Host "" + +$files = Get-WorkflowFiles +Write-Host "Found $($files.Count) workflow source files" -ForegroundColor Green + +$allFindings = @() + +foreach ($file in $files) { + $content = Get-Content -Path $file.FullName -Raw + $findings = Test-Workflow -Path $file.FullName -Content $content + + if ($findings.Count -gt 0) { + Write-Host " ⚠️ $($file.Name) — $($findings.Count) finding(s)" -ForegroundColor Yellow + foreach ($f in $findings) { + $icon = switch ($f.severity) { + "CRITICAL" { "🔴" } + "HIGH" { "🟡" } + "MODERATE" { "🟠" } + "LOW" { "🟢" } + default { "⚪" } + } + Write-Host " $icon [$($f.severity)] $($f.rule): $($f.message)" -ForegroundColor Gray + } + $allFindings += $findings + } + else { + Write-Host " ✅ $($file.Name)" -ForegroundColor Green + } +} + +Write-Host "`n=== Summary ===" -ForegroundColor Cyan + +if ($allFindings.Count -eq 0) { + Write-Host "✅ All workflows pass security checks" -ForegroundColor Green + exit 0 +} +else { + $bySeverity = $allFindings | Group-Object -Property severity + foreach ($g in $bySeverity) { + Write-Host " $($g.Name): $($g.Count)" -ForegroundColor Yellow + } + Write-Host "`n$($allFindings.Count) total finding(s)" -ForegroundColor Yellow + + $allFindings | ConvertTo-Json -Depth 5 + exit 1 +} diff --git a/.github/skills/instruction-drift/SKILL.md b/.github/skills/instruction-drift/SKILL.md index ea576212d..abb6cacff 100644 --- a/.github/skills/instruction-drift/SKILL.md +++ b/.github/skills/instruction-drift/SKILL.md @@ -1,74 +1,148 @@ --- name: instruction-drift description: >- - Detect and report staleness in instruction files that mirror upstream - documentation or cross-repo guidance. Use when checking whether - .github/instructions/*.md or .github/skills/*/SKILL.md are still in sync - with their declared upstream sources. Trigger words: "instruction drift", - "stale instructions", "sync check", "are instructions up to date", - "check freshness", "upstream changes". + Detect drift in instruction files and skills against upstream documentation sources. + Checks when tracked GitHub issues close, reference pages change, or new platform features + ship that aren't reflected locally. Use when asked to "check for drift", "are instructions up to date", + "instruction drift", "check if gh-aw docs are current", "update instructions from upstream", + "what changed upstream", or "have our instructions drifted". Also use proactively before major + instruction file edits to ensure you're working with current information. Produces a drift + report — never auto-edits files. --- -# Instruction-Drift Detection +# Instruction Drift Skill -This skill provides tooling to detect when instruction files (`.github/instructions/`, `.github/skills/`) have drifted from their declared upstream sources. +Detect when instruction files or skills have drifted from their upstream documentation sources. This skill **only detects and reports** — it never auto-edits instruction files. + +## When to Use + +- Before editing any instruction file that tracks external documentation +- When asked if instructions are up to date +- When upstream platforms (gh-aw, Helix, Android SDK) ship new releases +- Periodically (monthly) to catch drift ## How It Works -Each instruction or skill file that mirrors upstream content has a companion `.sync.yaml` file that declares: -- **target**: The local file to check -- **secondary_targets**: Additional files in the same skill directory -- **reference_urls**: Upstream documentation pages to compare against -- **tracked_issues**: GitHub issues whose resolution may require instruction updates -- **divergence_sections**: Sections that intentionally differ from upstream (repo-specific customizations) -- **releases_source**: GitHub releases feed for the upstream project +1. **Discover targets** — Find all `.sync.yaml` manifest files in the repository +2. **Snapshot sources** — Run `Check-Staleness.ps1` to capture current source status (issue states, page content hashes, latest releases with release notes) +3. **Crawl indexes** — Fetch doc site index pages (reference/, patterns/, guides/) and identify pages not yet tracked in the manifest +4. **Discover issues** — Find recently closed issues (last 90 days) in tracked repos that aren't in the manifest +5. **Flag drift signals** — Identify closed issues, fetch errors, content hash changes, untracked pages, untracked issues, and coverage gaps +6. **Report** — Present a prioritized drift report with actionable items for human review -## Running a Staleness Check +## Running the Skill -```powershell -# Check a specific sync manifest -pwsh .github/skills/instruction-drift/scripts/Check-Staleness.ps1 ` - -SyncManifest .github/instructions/gh-aw-workflows.sync.yaml +### Step 1: Run the staleness check script -# Check all sync manifests in the repo -Get-ChildItem -Recurse -Filter '*.sync.yaml' .github/ | - ForEach-Object { pwsh .github/skills/instruction-drift/scripts/Check-Staleness.ps1 -SyncManifest $_.FullName } +```bash +pwsh .github/skills/instruction-drift/scripts/Check-Staleness.ps1 ``` -## Sync Manifest Schema +The script outputs a JSON report to stdout with: +- Current state of tracked issues (open/closed) and whether resolution was expected +- Content hashes of reference pages (compare across runs to detect changes) +- Latest release tag and release notes (truncated to 2000 chars) for tracked repos +- **Untracked pages** — new doc pages discovered by crawling index pages that aren't in the manifest +- **Untracked closed issues** — recently closed issues (last 90 days) not yet tracked in the manifest +- Coverage gaps declared in the manifest (features from a page not yet documented locally) +- Whether any sources failed to fetch (404, timeout, etc.) +- `changes_detected` flag — true when any actionable signal is found + +### Step 2: Analyze the report + +Read the JSON output and the target instruction/skill files. For each detected signal: + +1. **Untracked pages** (highest priority) — New doc pages the manifest doesn't track. Fetch each page, determine if it documents features relevant to our skill, and if so: (a) add the URL to the manifest, (b) extract the key features, (c) update the skill files. +2. **Untracked closed issues** — Recently closed issues not in the manifest. Check if any represent feature ships or bug fixes that affect our documented guidance. Add relevant ones to the manifest with `resolution_expected: true`. +3. **Coverage gaps** — Features from a tracked page that we know we haven't documented. Fetch the page, extract the missing features, and update the skill files. +4. **Issue state change** — If a tracked issue closed, look for workarounds or "upstream issue" references that may now be obsolete. +5. **Page content change** — A hash changed but we don't know what specifically. Fetch the page and compare against what we document. Check `coverage_gaps` first — the change may have added features we already know we're missing. +6. **New release** — Read the release notes (included in the report) for features or breaking changes. Cross-reference against our anti-patterns table and feature documentation. +7. **Fetch failure** — A source URL returned 404 or timed out. The manifest may need updating (URL moved or docs restructured). + +### Step 3: Classify changes + +| Priority | Description | Action | +|----------|-------------|--------| +| **P0** | Factually wrong content (closed issue still referenced as open, deprecated feature documented as current) | Flag for immediate update | +| **P1** | Missing security-relevant changes (new anti-patterns, protection mechanisms) | Flag for prompt update | +| **P2** | Missing new features or capabilities | Flag for update when convenient | +| **P3** | Nice-to-have updates (new examples, reorganization) | Flag for future consideration | + +### Step 4: Output the report + +Present findings as a structured report. If invoked manually, show the report in the conversation. If invoked by a workflow, the report can be posted as a GitHub issue. + +## Manifest Format + +Each instruction file that tracks external sources has a `.sync.yaml` manifest co-located with it: ```yaml -# .github/instructions/example.sync.yaml -sync: - target: "../skills/my-skill/SKILL.md" - secondary_targets: - - "../skills/my-skill/references/deep-dive.md" - reference_urls: - - https://docs.example.com/guide - - https://docs.example.com/api - tracked_issues: - - url: https://github.com/org/repo/issues/123 - status: open - note: "Waiting for upstream fix" - divergence_sections: - - "Known Limitation: Custom Section" - - "Repo-Specific Configuration" - releases_source: https://github.com/org/repo/releases.atom - last_reviewed: "2025-07-01" +# .github/instructions/gh-aw-workflows.sync.yaml +target: ../skills/gh-aw-guide/SKILL.md + +# Sections that contain our own operational knowledge (not from upstream). +# The agent should NEVER remove or rewrite these — only append new content. +divergence: + - section: "Security Boundaries" + reason: "Our operational analysis and defense-in-depth assessment" + - section: "Safe Pattern: Checkout + Restore" + reason: "Documents our custom Checkout-GhAwPr.ps1 script" + +sources: + # Reference documentation pages + - url: https://github.github.com/gh-aw/reference/triggers/ + sections: ["Anti-Patterns", "Common Patterns", "Limitations"] + - url: https://github.github.com/gh-aw/reference/integrity/ + sections: ["Security Boundaries"] + coverage_gaps: # Features from this page NOT yet documented locally + - "endorsement-reactions (v0.68.2+)" + - "approval-labels for promoting items" + + # Tracked GitHub issues (check open/closed state) + - issue: github/gh-aw#18481 + resolution_expected: true + + # GitHub releases (check for new versions) + - releases: github/gh-aw + +style: | + Match existing section structure. Use tables for feature comparisons. ``` -## Output +### Source fields + +| Field | Required | Description | +|-------|----------|-------------| +| `url` | Yes (for web sources) | Reference page URL — script hashes content to detect changes | +| `sections` | No | Which local sections this source informs — helps the agent scope updates | +| `coverage_gaps` | No | Features from this page NOT yet documented locally — the agent should check these first when the page hash changes | +| `issue` | Yes (for issue sources) | Format: `owner/repo#number` — script checks open/closed state | +| `resolution_expected` | No | When `true` and issue closes, `changes_detected` is flagged | +| `releases` | Yes (for release sources) | Format: `owner/repo` — script checks latest release tag | -The script produces a structured report: -- **FRESH** (exit 0): All reference URLs respond 200, no tracked issues changed state vs. manifest, review window not exceeded -- **STALE** (exit 1): One or more signals — reference URLs unreachable, tracked issues changed state vs. manifest expected status, or review window exceeded -- **ERROR** (exit 2): Target file(s) declared in the manifest are missing from disk +## Important Constraints -Each signal includes actionable guidance on what to review and update. +- **Never auto-edit instruction files.** Always produce a drift report for human review. +- **Respect `divergence:` sections.** These contain hard-won operational knowledge that doesn't come from upstream docs. Never suggest removing this content. +- **Report fetch failures loudly.** A 404 on a tracked URL is a signal that the manifest needs updating, not that nothing changed. +- **Distinguish mechanical vs. judgment changes.** A closed issue is a mechanical fact. A restructured docs page requires human judgment about what to update. + +## Upstream Knowledge Extraction (Scan-GhAwUpdates.ps1) + +For the `gh-aw-guide` skill specifically, a second script mines the `github/gh-aw` repo directly: + +```powershell +pwsh .github/skills/instruction-drift/scripts/Scan-GhAwUpdates.ps1 +pwsh .github/skills/instruction-drift/scripts/Scan-GhAwUpdates.ps1 -MaxCommits 100 -DryRun +``` -## When to Run +This script: +1. **Tracks a watermark** (last checked commit SHA in `.gh-aw-watermark.json`) +2. **Fetches new commits** since the watermark +3. **Filters to high-signal changes** — features, safe-output additions, trigger changes, compiler updates, security fixes, breaking changes +4. **Extracts new features** categorized by type (safe-output, trigger, compiler, security, engine, breaking) +5. **Samples shared/ workflow configs** for real-world safe-output patterns +6. **Outputs a JSON knowledge report** with categorized changes -- Before any PR that modifies gh-aw workflow files -- Periodically (weekly recommended) to catch upstream documentation changes -- After any gh-aw platform release -- When a tracked upstream issue is closed +Use this when the Check-Staleness.ps1 reports that gh-aw docs pages have changed — Scan-GhAwUpdates tells you *what* changed at the commit level so you can update the skill with specific new features rather than re-reading entire doc pages. diff --git a/.github/skills/instruction-drift/scripts/Check-Staleness.ps1 b/.github/skills/instruction-drift/scripts/Check-Staleness.ps1 index 194b8782d..b06cafd45 100644 --- a/.github/skills/instruction-drift/scripts/Check-Staleness.ps1 +++ b/.github/skills/instruction-drift/scripts/Check-Staleness.ps1 @@ -1,218 +1,648 @@ <# .SYNOPSIS - Checks whether instruction/skill files are stale relative to their - declared upstream sources. + Checks instruction files for drift against upstream documentation sources. .DESCRIPTION - Reads a .sync.yaml manifest and checks: - 1. Target file(s) exist on disk - 2. Reference URLs are reachable (HTTP 200) - 3. Tracked issues have not changed status vs. manifest expected state - Reports FRESH, STALE, or ERROR with actionable details. - Note: releases_source is declared in the manifest schema but not yet - checked by this script — planned for a future enhancement. + Reads .sync.yaml manifest files and checks each configured source for changes: + - GitHub issues: checks open/closed state via gh CLI + - Web pages: fetches and computes content hash + - GitHub releases: checks latest release tag and release notes + - Index pages: crawls doc site indexes to discover new/untracked pages + - Recently closed issues: discovers closed issues not yet in the manifest -.PARAMETER SyncManifest - Path to the .sync.yaml file to check. + Outputs a JSON report to stdout describing what changed. -.PARAMETER Verbose - Show detailed output for each check. +.PARAMETER ManifestPath + Path to a specific .sync.yaml manifest. If not provided, discovers all manifests. + +.PARAMETER RepoRoot + Repository root directory. Defaults to current directory. .EXAMPLE - pwsh Check-Staleness.ps1 -SyncManifest .github/instructions/gh-aw-workflows.sync.yaml + pwsh Check-Staleness.ps1 + pwsh Check-Staleness.ps1 -ManifestPath .github/instructions/gh-aw-workflows.sync.yaml #> -[CmdletBinding()] param( - [Parameter(Mandatory = $true)] - [string]$SyncManifest + [string]$ManifestPath, + [string]$RepoRoot = (Get-Location).Path ) $ErrorActionPreference = 'Stop' -# --- Helpers --- +function Get-ContentHash { + param([string]$Content) + $bytes = [System.Text.Encoding]::UTF8.GetBytes($Content) + $sha = [System.Security.Cryptography.SHA256]::Create() + try { + $hash = $sha.ComputeHash($bytes) + } + finally { + $sha.Dispose() + } + return [System.BitConverter]::ToString($hash).Replace('-', '').ToLower().Substring(0, 16) +} + +function Test-GitHubIssue { + param( + [string]$Repo, + [int]$Number + ) -function Read-Yaml { - param([string]$Path) - # Minimal YAML parser for flat sync manifests — handles the subset we use. - # For full YAML, install powershell-yaml module. - $content = Get-Content -Path $Path -Raw - # Return raw content for manual parsing below - return $content + try { + $json = gh api "repos/$Repo/issues/$Number" --jq '{state: .state, title: .title, updated_at: .updated_at, closed_at: .closed_at}' 2>&1 + if ($LASTEXITCODE -ne 0) { + return @{ + status = 'error' + error = "Failed to fetch issue: $json" + number = $Number + repo = $Repo + } + } + $issue = $json | ConvertFrom-Json + return @{ + status = 'ok' + number = $Number + repo = $Repo + state = $issue.state + title = $issue.title + updated_at = $issue.updated_at + closed_at = $issue.closed_at + } + } + catch { + return @{ + status = 'error' + error = $_.Exception.Message + number = $Number + repo = $Repo + } + } } -function Test-UrlReachable { +function Test-WebPage { param([string]$Url) + try { - $response = Invoke-WebRequest -Uri $Url -Method Head -TimeoutSec 10 -UseBasicParsing -ErrorAction Stop - return @{ Url = $Url; Status = $response.StatusCode; Ok = ($response.StatusCode -eq 200) } + $response = Invoke-WebRequest -Uri $Url -UseBasicParsing -TimeoutSec 30 -ErrorAction Stop + $hash = Get-ContentHash -Content $response.Content + return @{ + status = 'ok' + url = $Url + content_hash = $hash + status_code = $response.StatusCode + } } catch { - return @{ Url = $Url; Status = $_.Exception.Message; Ok = $false } + $statusCode = 0 + if ($_.Exception.Response) { + $statusCode = [int]$_.Exception.Response.StatusCode + } + return @{ + status = 'error' + url = $Url + error = $_.Exception.Message + status_code = $statusCode + } } } -function Get-TrackedIssueStatus { - param([string]$Url) - # Extract owner/repo/number from GitHub issue URL - if ($Url -match 'github\.com/([^/]+)/([^/]+)/issues/(\d+)') { - $owner = $Matches[1] - $repo = $Matches[2] - $number = $Matches[3] - try { - $json = gh issue view $number --repo "$owner/$repo" --json state,title --jq '{state: .state, title: .title}' 2>&1 - if ($LASTEXITCODE -eq 0) { - $data = $json | ConvertFrom-Json - return @{ Url = $Url; State = $data.state; Title = $data.title; Ok = $true } +function Get-GitHubLatestRelease { + param([string]$Repo) + + try { + $json = gh api "repos/$Repo/releases?per_page=1" --jq '.[0] | {tag_name: .tag_name, name: .name, published_at: .published_at, body: .body}' 2>&1 + if ($LASTEXITCODE -ne 0) { + return @{ + status = 'error' + repo = $Repo + error = "Failed to fetch releases: $json" } } - catch { - Write-Verbose "Failed to check issue $Url`: $_" + if (-not $json -or $json -eq 'null') { + return @{ + status = 'ok' + repo = $Repo + latest = $null + } + } + $release = $json | ConvertFrom-Json + # Truncate release notes to first 2000 chars to keep report manageable + $body = if ($release.body.Length -gt 2000) { $release.body.Substring(0, 2000) + '...' } else { $release.body } + return @{ + status = 'ok' + repo = $Repo + latest = @{ + tag = $release.tag_name + name = $release.name + published_at = $release.published_at + release_notes = $body + } + } + } + catch { + return @{ + status = 'error' + repo = $Repo + error = $_.Exception.Message } } - return @{ Url = $Url; State = 'UNKNOWN'; Title = ''; Ok = $false } } -# --- Main --- +function Get-IndexPageLinks { + <# + .SYNOPSIS + Crawls an index page and extracts all documentation links. + .DESCRIPTION + Fetches an index/section page from a docs site and extracts relative + links to child pages. Used to discover new pages not yet in the manifest. + #> + param( + [string]$IndexUrl, + [string]$BaseUrl + ) -if (-not (Test-Path $SyncManifest)) { - Write-Error "Sync manifest not found: $SyncManifest" - exit 1 -} + try { + $response = Invoke-WebRequest -Uri $IndexUrl -UseBasicParsing -TimeoutSec 30 -ErrorAction Stop + $content = $response.Content -$manifestDir = Split-Path -Parent (Resolve-Path $SyncManifest) -$raw = Get-Content -Path $SyncManifest -Raw + # Extract href links that are relative paths (not external, not anchors) + $links = @() + $hrefPattern = 'href="([^"#]+)"' + $matches_ = [regex]::Matches($content, $hrefPattern) + foreach ($m in $matches_) { + $href = $m.Groups[1].Value + # Skip external links, assets, anchors, and non-doc paths + if ($href -match '^(https?://|mailto:|#|/assets/|/images/)') { continue } + if ($href -match '\.(css|js|png|jpg|svg|ico|xml|json)$') { continue } -Write-Host "=== Instruction Drift Check ===" -ForegroundColor Cyan -Write-Host "Manifest: $SyncManifest" -Write-Host "" + # Resolve relative to index URL using System.Uri for correct path handling + $resolvedUri = [System.Uri]::new([System.Uri]::new($IndexUrl), $href) + $resolvedUrl = $resolvedUri.AbsoluteUri -$signals = @() -$errors = @() + # Normalize: remove trailing slashes for comparison, then add back + $resolvedUrl = $resolvedUrl.TrimEnd('/') + '/' + if ($resolvedUrl -ne $IndexUrl -and $resolvedUrl.StartsWith($BaseUrl)) { + $links += $resolvedUrl + } + } -# Check target file exists -if ($raw -match 'target:\s*"([^"]+)"') { - $targetPath = Join-Path $manifestDir $Matches[1] - if (Test-Path $targetPath) { - Write-Host "✅ Target exists: $targetPath" -ForegroundColor Green + return @{ + status = 'ok' + url = $IndexUrl + links = ($links | Sort-Object -Unique) + } } - else { - Write-Host "❌ Target MISSING: $targetPath" -ForegroundColor Red - $errors += "Target file missing: $targetPath" + catch { + return @{ + status = 'error' + url = $IndexUrl + error = $_.Exception.Message + } } } -# Check secondary targets -$secondaryMatches = [regex]::Matches($raw, 'secondary_targets:\s*\n((?:\s*-\s*"[^"]+"\s*\n?)+)') -if ($secondaryMatches.Count -gt 0) { - $secondaryPaths = [regex]::Matches($secondaryMatches[0].Value, '"([^"]+)"') - foreach ($m in $secondaryPaths) { - $secPath = Join-Path $manifestDir $m.Groups[1].Value - if (Test-Path $secPath) { - Write-Host "✅ Secondary target exists: $secPath" -ForegroundColor Green +function Find-UntrackedPages { + <# + .SYNOPSIS + Compares discovered index links against tracked URLs in the manifest. + .DESCRIPTION + Crawls doc site index pages and identifies pages not yet tracked + in any manifest source. Returns a list of untracked URLs. + #> + param( + [string[]]$IndexUrls, + [string]$BaseUrl, + [string[]]$TrackedUrls + ) + + $allDiscovered = @() + foreach ($indexUrl in $IndexUrls) { + Write-Host " 🔎 Crawling index: $indexUrl..." -NoNewline + $result = Get-IndexPageLinks -IndexUrl $indexUrl -BaseUrl $BaseUrl + if ($result.status -eq 'ok') { + Write-Host " found $($result.links.Count) links" -ForegroundColor Green + $allDiscovered += $result.links } else { - Write-Host "❌ Secondary target MISSING: $secPath" -ForegroundColor Red - $errors += "Secondary target missing: $secPath" + Write-Host " ❌ $($result.error)" -ForegroundColor Red } } + + $allDiscovered = $allDiscovered | Sort-Object -Unique + + # Normalize tracked URLs for comparison + $normalizedTracked = $TrackedUrls | ForEach-Object { $_.TrimEnd('/') + '/' } + + $untracked = $allDiscovered | Where-Object { + $normalized = $_.TrimEnd('/') + '/' + $normalized -notin $normalizedTracked + } + + return $untracked } -# Check reference URLs -$urlMatches = [regex]::Matches($raw, 'reference_urls:\s*\n((?:\s*-\s*https?://[^\s]+\s*\n?)+)') -if ($urlMatches.Count -gt 0) { - $urls = [regex]::Matches($urlMatches[0].Value, '(https?://[^\s]+)') - Write-Host "`nChecking reference URLs..." -ForegroundColor Cyan - foreach ($u in $urls) { - $result = Test-UrlReachable -Url $u.Groups[1].Value - if ($result.Ok) { - Write-Host " ✅ $($result.Url)" -ForegroundColor Green +function Get-RecentClosedIssues { + <# + .SYNOPSIS + Discovers recently closed issues in a repo not yet tracked in the manifest. + .DESCRIPTION + Fetches issues closed in the last 90 days and identifies ones + not already in the manifest's tracked issues list. + #> + param( + [string]$Repo, + [int[]]$TrackedIssueNumbers, + [int]$DaysBack = 90 + ) + + try { + $since = (Get-Date).AddDays(-$DaysBack).ToString('yyyy-MM-ddTHH:mm:ssZ') + # --paginate feeds each page through jq separately, so emit objects + # (not wrapped in array) to allow clean concatenation across pages. + # Wrap the entire output in [...] for ConvertFrom-Json. + $raw = gh api --paginate "repos/$Repo/issues?state=closed&since=$since&per_page=100&sort=updated&direction=desc" --jq '.[] | select(.pull_request == null) | {number: .number, title: .title, closed_at: .closed_at, labels: [.labels[].name]}' 2>&1 + if ($LASTEXITCODE -ne 0) { + return @{ + status = 'error' + repo = $Repo + error = "Failed to fetch issues: $raw" + } + } + if (-not $raw -or ($raw -is [array] -and $raw.Count -eq 0) -or ([string]::IsNullOrWhiteSpace(($raw -join '')))) { + $issues = @() } else { - Write-Host " ⚠️ $($result.Url) — $($result.Status)" -ForegroundColor Yellow - $signals += "Reference URL unreachable (may have moved or been removed): $($result.Url)" + # $raw is an Object[] of JSON lines from gh --paginate --jq. + # Join into a comma-separated array for ConvertFrom-Json. + $lines = @($raw) | Where-Object { $_ -and $_.Trim() -ne '' } + $jsonArray = "[" + ($lines -join ",") + "]" + $issues = $jsonArray | ConvertFrom-Json + } + $untracked = $issues | Where-Object { $_.number -notin $TrackedIssueNumbers } + return @{ + status = 'ok' + repo = $Repo + untracked = @($untracked) + total_checked = ($issues | Measure-Object).Count + } + } + catch { + return @{ + status = 'error' + repo = $Repo + error = $_.Exception.Message } } } -# Check tracked issues — compare actual state vs. manifest expected state -$issueMatches = [regex]::Matches($raw, 'url:\s*(https://github\.com/[^\s]+)\s*\n\s*status:\s*(\w+)') -if ($issueMatches.Count -gt 0) { - Write-Host "`nChecking tracked issues..." -ForegroundColor Cyan - foreach ($im in $issueMatches) { - $issueUrl = $im.Groups[1].Value - $expectedStatus = $im.Groups[2].Value.ToUpper() - $issueResult = Get-TrackedIssueStatus -Url $issueUrl - if ($issueResult.Ok) { - $actualState = $issueResult.State.ToUpper() - $stateIcon = if ($actualState -eq 'CLOSED') { '🔒' } else { '🔓' } - if ($actualState -eq $expectedStatus) { - Write-Host " $stateIcon $($issueResult.Url) — $($issueResult.State) (matches expected)" -ForegroundColor Green - } - elseif ($actualState -eq 'CLOSED' -and $expectedStatus -eq 'OPEN') { - Write-Host " $stateIcon $($issueResult.Url) — CLOSED (was expected OPEN): $($issueResult.Title)" -ForegroundColor Yellow - $signals += "Tracked issue just CLOSED — may need instruction update: $($issueResult.Url)" - } - elseif ($actualState -eq 'OPEN' -and $expectedStatus -eq 'CLOSED') { - Write-Host " $stateIcon $($issueResult.Url) — REOPENED (was expected CLOSED): $($issueResult.Title)" -ForegroundColor Yellow - $signals += "Tracked issue REOPENED — was recorded as closed: $($issueResult.Url)" +function ConvertFrom-SyncManifest { + param([string]$Path) + + # Simple YAML parser for our specific manifest format + $content = Get-Content $Path -Raw + $manifest = @{ + target = '' + secondary_targets = @() + sources = @() + divergence = @() + path = $Path + } + + $lines = $content -split "`n" + $currentSection = '' + $currentItem = $null + + foreach ($line in $lines) { + $trimmed = $line.Trim() + + # Skip comments and empty lines + if ($trimmed -eq '' -or $trimmed.StartsWith('#')) { continue } + + if ($trimmed -match '^target:\s*(.+)$') { + $manifest.target = $Matches[1].Trim() + } + elseif ($trimmed -eq 'secondary_targets:') { + $currentSection = 'secondary_targets' + } + elseif ($currentSection -eq 'secondary_targets') { + if ($trimmed -match '^-\s*(.+)$') { + $manifest.secondary_targets += $Matches[1].Trim() } else { - Write-Host " ⚠️ $($issueResult.Url) — $actualState (expected $expectedStatus)" -ForegroundColor Yellow - $signals += "Tracked issue state mismatch ($actualState vs expected $expectedStatus): $($issueResult.Url)" + # Not a list item — fall through to other section checks below + $currentSection = '' + # Re-check this line against section headers + if ($trimmed -eq 'sources:') { $currentSection = 'sources' } + elseif ($trimmed -eq 'divergence:') { $currentSection = 'divergence' } + elseif ($trimmed -eq 'style: |') { $currentSection = 'style' } } } - else { - Write-Host " ❓ $issueUrl — could not check" -ForegroundColor Yellow + elseif ($trimmed -eq 'sources:') { + $currentSection = 'sources' + } + elseif ($trimmed -eq 'divergence:') { + $currentSection = 'divergence' + } + elseif ($trimmed -eq 'style: |') { + $currentSection = 'style' + } + elseif ($currentSection -eq 'sources') { + if ($trimmed -match '^-\s*url:\s*(.+)$') { + $currentItem = @{ type = 'web'; url = $Matches[1].Trim() } + $manifest.sources += $currentItem + } + elseif ($trimmed -match '^-\s*issue:\s*(.+)$') { + $issueRef = $Matches[1].Trim() + if ($issueRef -match '^(.+)#(\d+)$') { + $currentItem = @{ + type = 'issue' + repo = $Matches[1] + number = [int]$Matches[2] + } + $manifest.sources += $currentItem + } + } + elseif ($trimmed -match '^-\s*releases:\s*(.+)$') { + $currentItem = @{ type = 'releases'; repo = $Matches[1].Trim() } + $manifest.sources += $currentItem + } + elseif ($trimmed -match '^resolution_expected:\s*(.+)$') { + if ($currentItem) { + $currentItem.resolution_expected = $Matches[1].Trim() -eq 'true' + } + } + elseif ($trimmed -match '^coverage_gaps:') { + if ($currentItem) { + $currentItem.coverage_gaps = @() + $currentSection = 'coverage_gaps' + } + } + elseif ($trimmed -match '^sections:') { + # sections are informational, pass through + } + } + elseif ($currentSection -eq 'coverage_gaps') { + # Source-entry patterns or section transition — exit coverage_gaps + if ($trimmed -match '^-\s*(url|issue|releases):' -or + $trimmed -eq 'divergence:' -or $trimmed -eq 'style: |' -or + ($trimmed -ne '' -and -not $trimmed.StartsWith('-'))) { + # Left coverage_gaps block — determine correct section + if ($trimmed -eq 'divergence:') { + $currentSection = 'divergence' + } + elseif ($trimmed -eq 'style: |') { + $currentSection = 'style' + } + else { + $currentSection = 'sources' + # Re-process this line in sources context + if ($trimmed -match '^-\s*url:\s*(.+)$') { + $currentItem = @{ type = 'web'; url = $Matches[1].Trim() } + $manifest.sources += $currentItem + } + elseif ($trimmed -match '^-\s*issue:\s*(.+)$') { + $issueRef = $Matches[1].Trim() + if ($issueRef -match '^(.+)#(\d+)$') { + $currentItem = @{ type = 'issue'; repo = $Matches[1]; number = [int]$Matches[2] } + $manifest.sources += $currentItem + } + } + elseif ($trimmed -match '^-\s*releases:\s*(.+)$') { + $currentItem = @{ type = 'releases'; repo = $Matches[1].Trim() } + $manifest.sources += $currentItem + } + } + } + elseif ($trimmed -match '^-\s*"?(.+?)"?$') { + if ($currentItem -and $currentItem.ContainsKey('coverage_gaps')) { + $currentItem.coverage_gaps += $Matches[1] + } + } + } + elseif ($currentSection -eq 'divergence') { + if ($trimmed -match '^-\s*section:\s*"?(.+?)"?$') { + $currentItem = @{ section = $Matches[1] } + $manifest.divergence += $currentItem + } + elseif ($trimmed -match '^reason:\s*"?(.+?)"?$') { + if ($currentItem) { + $currentItem.reason = $Matches[1] + } + } } } + + return $manifest +} + +# --- Main --- + +Write-Host "🔍 Checking instruction file staleness..." -ForegroundColor Cyan + +# Discover manifests +$manifests = @() +if ($ManifestPath) { + $fullPath = Join-Path $RepoRoot $ManifestPath + if (-not (Test-Path $fullPath)) { + Write-Error "Manifest not found: $fullPath" + exit 1 + } + $manifests += $fullPath } -# Fallback: issues without status field (legacy format) -# Skip URLs already captured by the primary regex above to avoid truncated matches -$primaryUrls = @() -foreach ($pm in $issueMatches) { $primaryUrls += $pm.Groups[1].Value } -$allIssueUrls = [regex]::Matches($raw, '-\s*url:\s*(https://github\.com/[^\s]+)') -if ($allIssueUrls.Count -gt 0) { - foreach ($au in $allIssueUrls) { - $issueUrl = $au.Groups[1].Value - if ($primaryUrls -contains $issueUrl) { continue } - $issueResult = Get-TrackedIssueStatus -Url $issueUrl - if ($issueResult.Ok -and $issueResult.State -eq 'CLOSED') { - Write-Host " 🔒 $($issueResult.Url) — CLOSED (no expected status declared): $($issueResult.Title)" -ForegroundColor Yellow - $signals += "Tracked issue CLOSED (no expected status in manifest): $($issueResult.Url)" +else { + # Use .github/ as primary search path since manifests live next to instruction files. + # Also search repo root for any other manifests. + $searchPaths = @( + (Join-Path $RepoRoot '.github') + ) + $manifests = @() + foreach ($sp in $searchPaths) { + if (Test-Path $sp) { + $manifests += Get-ChildItem -Path $sp -Filter '*.sync.yaml' -Recurse -Depth 4 -ErrorAction SilentlyContinue | + Select-Object -ExpandProperty FullName } } } -# Check last_reviewed date -if ($raw -match 'last_reviewed:\s*"(\d{4}-\d{2}-\d{2})"') { - $lastReviewed = [datetime]::Parse($Matches[1]) - $daysSince = ([datetime]::UtcNow - $lastReviewed).Days - Write-Host "`nLast reviewed: $($Matches[1]) ($daysSince days ago)" -ForegroundColor Cyan - if ($daysSince -gt 30) { - $signals += "Last reviewed $daysSince days ago (threshold: 30 days)" - Write-Host " ⚠️ Over 30 days since last review" -ForegroundColor Yellow +if ($manifests.Count -eq 0) { + Write-Host "No .sync.yaml manifests found." -ForegroundColor Yellow + @{ manifests = @(); changes_detected = $false } | ConvertTo-Json -Depth 10 + exit 0 +} + +Write-Host "Found $($manifests.Count) manifest(s)" -ForegroundColor Green + +$results = @() + +foreach ($manifestPath in $manifests) { + Write-Host "`n📋 Processing: $manifestPath" -ForegroundColor Cyan + + $manifest = ConvertFrom-SyncManifest -Path $manifestPath + Write-Host " Target: $($manifest.target)" + Write-Host " Sources: $($manifest.sources.Count)" + + $sourceResults = @() + + foreach ($source in $manifest.sources) { + switch ($source.type) { + 'issue' { + Write-Host " 🔗 Checking issue $($source.repo)#$($source.number)..." -NoNewline + $result = Test-GitHubIssue -Repo $source.repo -Number $source.number + $resExpected = if ($source.ContainsKey('resolution_expected')) { $source.resolution_expected } else { $false } + if ($result.status -eq 'ok') { + # Compare actual state against expected state + $expectedState = if ($resExpected) { 'closed' } else { 'open' } + $stateMatch = $result.state -eq $expectedState + $stateEmoji = if ($stateMatch) { '✅' } else { '⚠️' } + $stateColor = if ($stateMatch) { 'Green' } else { 'Yellow' } + Write-Host " $stateEmoji $($result.state) (expected: $expectedState)" -ForegroundColor $stateColor + } + else { + Write-Host " ❌ Error: $($result.error)" -ForegroundColor Red + } + $sourceResults += @{ + type = 'issue' + ref = "$($source.repo)#$($source.number)" + resolution_expected = $resExpected + result = $result + } + } + 'web' { + Write-Host " 🌐 Checking $($source.url)..." -NoNewline + $result = Test-WebPage -Url $source.url + $gaps = if ($source.ContainsKey('coverage_gaps')) { $source.coverage_gaps } else { @() } + if ($result.status -eq 'ok') { + Write-Host " ✅ hash=$($result.content_hash)" -ForegroundColor Green + } + else { + Write-Host " ❌ Error: $($result.error)" -ForegroundColor Red + } + if ($gaps.Count -gt 0) { + Write-Host " ⚠️ $($gaps.Count) known coverage gap(s)" -ForegroundColor Yellow + } + $entry = @{ + type = 'web' + url = $source.url + result = $result + } + if ($gaps.Count -gt 0) { + $entry.coverage_gaps = $gaps + } + $sourceResults += $entry + } + 'releases' { + Write-Host " 📦 Checking releases for $($source.repo)..." -NoNewline + $result = Get-GitHubLatestRelease -Repo $source.repo + if ($result.status -eq 'ok' -and $result.latest) { + Write-Host " ✅ latest=$($result.latest.tag)" -ForegroundColor Green + } + elseif ($result.status -eq 'ok') { + Write-Host " ℹ️ No releases found" -ForegroundColor Yellow + } + else { + Write-Host " ❌ Error: $($result.error)" -ForegroundColor Red + } + $sourceResults += @{ + type = 'releases' + repo = $source.repo + result = $result + } + } + } + } + + $resultEntry = @{ + manifest = [System.IO.Path]::GetRelativePath($RepoRoot, $manifestPath) + target = $manifest.target + sources = $sourceResults } - else { - Write-Host " ✅ Within 30-day review window" -ForegroundColor Green + if ($manifest.secondary_targets.Count -gt 0) { + $resultEntry.secondary_targets = $manifest.secondary_targets + } + $results += $resultEntry + + # --- Discovery: find untracked pages and recently closed issues --- + + # Collect tracked URLs and issue numbers from this manifest + $trackedUrls = @($manifest.sources | Where-Object { $_.type -eq 'web' } | ForEach-Object { $_.url }) + $trackedIssueNumbers = @($manifest.sources | Where-Object { $_.type -eq 'issue' } | ForEach-Object { $_.number }) + $releaseRepos = @($manifest.sources | Where-Object { $_.type -eq 'releases' } | ForEach-Object { $_.repo }) + $issueRepos = @($manifest.sources | Where-Object { $_.type -eq 'issue' } | ForEach-Object { $_.repo } | Sort-Object -Unique) + $allRepos = @($releaseRepos + $issueRepos | Sort-Object -Unique) + + # Discover base URL from tracked URLs and crawl index pages + $baseUrls = $trackedUrls | ForEach-Object { + if ($_ -match '^(https://[^/]+/[^/]+/)') { $Matches[1] } + } | Sort-Object -Unique + + foreach ($baseUrl in $baseUrls) { + Write-Host "`n🔍 Discovering new pages under $baseUrl" -ForegroundColor Cyan + + # Common doc site section indexes to crawl + $indexUrls = @( + "${baseUrl}reference/" + "${baseUrl}patterns/" + "${baseUrl}guides/" + ) + + $untrackedPages = Find-UntrackedPages -IndexUrls $indexUrls -BaseUrl $baseUrl -TrackedUrls $trackedUrls + if ($untrackedPages.Count -gt 0) { + Write-Host " ⚠️ Found $($untrackedPages.Count) untracked page(s):" -ForegroundColor Yellow + foreach ($page in $untrackedPages) { + Write-Host " 📄 $page" -ForegroundColor Yellow + } + # Add to results + $results[-1].untracked_pages = @($untrackedPages) + } + else { + Write-Host " ✅ All discovered pages are tracked" -ForegroundColor Green + } } -} -# Summary -Write-Host "`n=== Summary ===" -ForegroundColor Cyan -if ($errors.Count -gt 0) { - Write-Host "Status: ERROR" -ForegroundColor Red - foreach ($e in $errors) { Write-Host " ❌ $e" -ForegroundColor Red } - exit 2 + # Discover recently closed issues not in the manifest + foreach ($repo in $allRepos) { + Write-Host "`n🔍 Checking recently closed issues in $repo" -ForegroundColor Cyan + # Filter tracked issue numbers to THIS repo to avoid cross-repo collisions + $repoTrackedNumbers = @($manifest.sources | Where-Object { $_.type -eq 'issue' -and $_.repo -eq $repo } | ForEach-Object { $_.number }) + $closedResult = Get-RecentClosedIssues -Repo $repo -TrackedIssueNumbers $repoTrackedNumbers + if ($closedResult.status -eq 'ok') { + $untrackedCount = ($closedResult.untracked | Measure-Object).Count + if ($untrackedCount -gt 0) { + Write-Host " ⚠️ Found $untrackedCount recently closed issue(s) not in manifest:" -ForegroundColor Yellow + foreach ($issue in $closedResult.untracked) { + $labels = if ($issue.labels) { " [$($issue.labels -join ', ')]" } else { '' } + Write-Host " #$($issue.number): $($issue.title)$labels" -ForegroundColor Yellow + } + $results[-1].untracked_closed_issues = @($closedResult.untracked) + } + else { + Write-Host " ✅ No new closed issues (checked $($closedResult.total_checked) in last 90 days)" -ForegroundColor Green + } + } + else { + Write-Host " ❌ $($closedResult.error)" -ForegroundColor Red + } + } } -elseif ($signals.Count -gt 0) { - Write-Host "Status: STALE" -ForegroundColor Yellow - foreach ($s in $signals) { Write-Host " ⚠️ $s" -ForegroundColor Yellow } - Write-Host "`nAction: Review the signals above and update instructions if needed." -ForegroundColor Yellow - exit 1 + +# Output JSON report +# changes_detected flags sources that need attention: +# - fetch errors (source may have moved) +# - closed issues where resolution_expected is true (instruction may reference outdated workarounds) +# - untracked pages discovered via index crawling +# - untracked recently closed issues +$actionableChanges = $results | ForEach-Object { $_.sources } | Where-Object { + $_.result.status -eq 'error' -or + ($_.type -eq 'issue' -and $_.resolution_expected -and $_.result.status -eq 'ok' -and $_.result.state -eq 'closed') } -else { - Write-Host "Status: FRESH ✅" -ForegroundColor Green - Write-Host "All checks passed — instructions appear up to date." -ForegroundColor Green - exit 0 +$hasUntrackedPages = ($results | Where-Object { $_.untracked_pages.Count -gt 0 } | Measure-Object).Count -gt 0 +$hasUntrackedIssues = ($results | Where-Object { $_.untracked_closed_issues.Count -gt 0 } | Measure-Object).Count -gt 0 +$report = @{ + checked_at = (Get-Date -Format 'o') + manifests = $results + changes_detected = (($actionableChanges | Measure-Object).Count -gt 0) -or $hasUntrackedPages -or $hasUntrackedIssues } + +Write-Host "`n📊 Report:" -ForegroundColor Cyan +$report | ConvertTo-Json -Depth 10 diff --git a/.github/skills/instruction-drift/scripts/Scan-GhAwUpdates.ps1 b/.github/skills/instruction-drift/scripts/Scan-GhAwUpdates.ps1 new file mode 100644 index 000000000..d60992686 --- /dev/null +++ b/.github/skills/instruction-drift/scripts/Scan-GhAwUpdates.ps1 @@ -0,0 +1,324 @@ +<# +.SYNOPSIS + Extracts gh-aw knowledge from the github/gh-aw repo by scanning + recent commits for high-signal changes to docs, compiler, and + safe-outputs infrastructure. + +.DESCRIPTION + Tracks a watermark (last checked commit SHA) and on each run: + 1. Fetches commits since the watermark + 2. Filters to high-signal commits (docs, features, safe-outputs, + triggers, security, compiler, breaking changes) + 3. For each high-signal commit, fetches the diff and extracts + relevant changes (new safe-output types, new frontmatter fields, + new anti-patterns, breaking changes) + 4. Scans shared/ workflow configs for real-world safe-output patterns + 5. Outputs a JSON knowledge update report + 6. Updates the watermark + +.PARAMETER WatermarkFile + Path to the watermark file (stores last checked SHA + timestamp). + Default: .github/skills/gh-aw-guide/.gh-aw-watermark.json + +.PARAMETER MaxCommits + Maximum number of commits to scan per run. Default: 50. + +.PARAMETER DryRun + If set, don't update the watermark file. + +.EXAMPLE + pwsh Scan-GhAwUpdates.ps1 + pwsh Scan-GhAwUpdates.ps1 -MaxCommits 100 -DryRun +#> + +[CmdletBinding()] +param( + [string]$WatermarkFile = ".github/skills/gh-aw-guide/.gh-aw-watermark.json", + [int]$MaxCommits = 50, + [switch]$DryRun +) + +$ErrorActionPreference = 'Stop' +$Repo = "github/gh-aw" + +# High-signal commit patterns — only these are worth extracting +$HighSignalPatterns = @( + 'feat:', + 'BREAKING', + 'safe.output', 'safe-output', + 'trigger', 'slash.command', 'label.command', + 'compiler', + 'security', 'integrity', 'protected.file', + 'merge-pull-request', + 'engine', + '^docs:.*(?:reference|pattern|guide|troubleshoot)' +) +$HighSignalRegex = ($HighSignalPatterns -join '|') + +# File path patterns that indicate knowledge-relevant changes +$KnowledgePaths = @( + '^docs/', + '^actions/safe_outputs/', + '^actions/.*\.cjs$', + 'schema', + '^\.github/workflows/shared/' +) +$KnowledgePathRegex = ($KnowledgePaths -join '|') + +function Get-Watermark { + if (Test-Path $WatermarkFile) { + $content = Get-Content $WatermarkFile -Raw | ConvertFrom-Json + return $content + } + return @{ + last_sha = $null + last_checked = $null + known_features = @() + } +} + +function Save-Watermark { + param($Watermark) + if (-not $DryRun) { + $dir = Split-Path -Parent $WatermarkFile + if (-not (Test-Path $dir)) { New-Item -ItemType Directory -Path $dir -Force | Out-Null } + $Watermark | ConvertTo-Json -Depth 5 | Set-Content -Path $WatermarkFile + } +} + +function Get-CommitsSinceWatermark { + param([string]$SinceShа) + + $args_ = @("api", "repos/$Repo/commits", "--per-page", $MaxCommits) + if ($SinceShа) { + # Use since parameter with the commit's date + $dateJson = gh api "repos/$Repo/commits/$SinceShа" --jq '.commit.committer.date' 2>&1 + if ($LASTEXITCODE -eq 0 -and $dateJson) { + $args_ += @("--jq", ".[].sha") + $args_ += @("-f", "since=$dateJson") + } + } + + $shas = gh @args_ 2>&1 + if ($LASTEXITCODE -ne 0) { + Write-Warning "Failed to fetch commits: $shas" + return @() + } + + $commitList = $shas -split "`n" | Where-Object { $_ -and $_ -ne $SinceShа } + return $commitList +} + +function Get-CommitInfo { + param([string]$Sha) + $json = gh api "repos/$Repo/commits/$Sha" --jq '{ + sha: .sha, + message: .commit.message, + date: .commit.committer.date, + files: [.files[] | {filename, status, additions, deletions}] + }' 2>&1 + + if ($LASTEXITCODE -ne 0) { return $null } + return $json | ConvertFrom-Json +} + +function Test-HighSignal { + param($CommitMessage) + return $CommitMessage -match $HighSignalRegex +} + +function Test-KnowledgePath { + param([string]$Path) + return $Path -match $KnowledgePathRegex +} + +function Extract-SafeOutputPatterns { + <# + .SYNOPSIS + Scans shared/ workflow configs for real-world safe-output patterns. + #> + Write-Host "📦 Scanning shared workflow configs for safe-output patterns..." -ForegroundColor Cyan + + $sharedFiles = gh api "repos/$Repo/git/trees/main?recursive=1" --jq ' + [.tree[] | select(.path | test("^.github/workflows/shared/.*\\.md$")) | .path] + ' 2>&1 | ConvertFrom-Json + + $patterns = @() + $sampleCount = [Math]::Min(20, $sharedFiles.Count) + $sampled = $sharedFiles | Get-Random -Count $sampleCount + + foreach ($file in $sampled) { + $content = gh api "repos/$Repo/contents/$file" --jq '.content' 2>&1 + if ($LASTEXITCODE -ne 0) { continue } + + $decoded = [System.Text.Encoding]::UTF8.GetString([Convert]::FromBase64String($content)) + + # Extract YAML frontmatter + if ($decoded -match '(?s)^---\s*\n(.*?)\n---') { + $frontmatter = $Matches[1] + + # Look for safe-outputs block + if ($frontmatter -match '(?s)safe-outputs:\s*\n(.+?)(?=\n\S|\z)') { + $patterns += @{ + file = $file + safe_outputs = $Matches[1].Trim() + } + } + } + } + + return $patterns +} + +function Extract-NewFeatures { + param($Commit) + + $features = @() + $message = $Commit.message + $firstLine = ($message -split "`n")[0] + + # Detect new safe-output types + if ($message -match 'safe.output|safe-output') { + $features += @{ + type = "safe-output" + summary = $firstLine + sha = $Commit.sha.Substring(0, 8) + } + } + + # Detect new trigger types + if ($message -match 'trigger|slash.command|label.command') { + $features += @{ + type = "trigger" + summary = $firstLine + sha = $Commit.sha.Substring(0, 8) + } + } + + # Detect compiler changes + if ($message -match 'compiler|compile') { + $features += @{ + type = "compiler" + summary = $firstLine + sha = $Commit.sha.Substring(0, 8) + } + } + + # Detect security changes + if ($message -match 'security|integrity|protected.file|XPIA') { + $features += @{ + type = "security" + summary = $firstLine + sha = $Commit.sha.Substring(0, 8) + } + } + + # Detect breaking changes + if ($message -match 'BREAKING|breaking.change') { + $features += @{ + type = "breaking" + summary = $firstLine + sha = $Commit.sha.Substring(0, 8) + } + } + + # Detect new engine support + if ($message -match 'engine|OpenCode|Codex|Claude') { + $features += @{ + type = "engine" + summary = $firstLine + sha = $Commit.sha.Substring(0, 8) + } + } + + return $features +} + +# --- Main --- + +Write-Host "🔍 gh-aw Knowledge Extraction" -ForegroundColor Cyan +Write-Host "Repository: $Repo" +Write-Host "" + +$watermark = Get-Watermark +if ($watermark.last_sha) { + Write-Host "Watermark: $($watermark.last_sha.Substring(0, 8)) ($($watermark.last_checked))" -ForegroundColor Green +} else { + Write-Host "No watermark — first run, scanning last $MaxCommits commits" -ForegroundColor Yellow +} + +# Fetch commits +$commits = Get-CommitsSinceWatermark -SinceShа $watermark.last_sha +Write-Host "Found $($commits.Count) new commits" -ForegroundColor Cyan + +if ($commits.Count -eq 0) { + Write-Host "✅ No new commits since last check" -ForegroundColor Green + @{ checked_at = (Get-Date -Format 'o'); new_features = @(); changes_detected = $false } | ConvertTo-Json -Depth 5 + exit 0 +} + +# Filter to high-signal commits +$highSignalCommits = @() +$allFeatures = @() + +foreach ($sha in $commits) { + $info = Get-CommitInfo -Sha $sha + if (-not $info) { continue } + + $firstLine = ($info.message -split "`n")[0] + + if (Test-HighSignal -CommitMessage $info.message) { + $hasKnowledgeFiles = $info.files | Where-Object { Test-KnowledgePath -Path $_.filename } + + Write-Host " ⚡ $($sha.Substring(0, 8)) $firstLine" -ForegroundColor Yellow + $highSignalCommits += $info + + $features = Extract-NewFeatures -Commit $info + $allFeatures += $features + } +} + +Write-Host "`n📊 High-signal commits: $($highSignalCommits.Count) / $($commits.Count)" -ForegroundColor Cyan + +# Extract safe-output patterns from shared configs +$safeOutputPatterns = Extract-SafeOutputPatterns + +# Categorize features +$byType = $allFeatures | Group-Object -Property type + +Write-Host "`n=== Knowledge Update Summary ===" -ForegroundColor Cyan +foreach ($group in $byType) { + Write-Host " $($group.Name): $($group.Count) changes" -ForegroundColor Yellow + foreach ($f in $group.Group) { + Write-Host " $($f.sha) $($f.summary)" -ForegroundColor Gray + } +} + +# Build report +$report = @{ + checked_at = (Get-Date -Format 'o') + watermark_from = $watermark.last_sha + watermark_to = $commits[0] + commits_scanned = $commits.Count + high_signal_count = $highSignalCommits.Count + new_features = $allFeatures + safe_output_samples = $safeOutputPatterns + changes_detected = ($allFeatures.Count -gt 0) + feature_summary = @{} +} + +foreach ($group in $byType) { + $report.feature_summary[$group.Name] = @{ + count = $group.Count + items = $group.Group | ForEach-Object { "$($_.sha) $($_.summary)" } + } +} + +# Update watermark +$watermark.last_sha = $commits[0] +$watermark.last_checked = (Get-Date -Format 'o') +$watermark.known_features += $allFeatures | ForEach-Object { "$($_.type):$($_.sha)" } +Save-Watermark -Watermark $watermark + +Write-Host "`n📄 Report:" -ForegroundColor Cyan +$report | ConvertTo-Json -Depth 10 diff --git a/.github/workflows/dep-update.lock.yml b/.github/workflows/dep-update.lock.yml index 2f8390f1b..912249b48 100644 --- a/.github/workflows/dep-update.lock.yml +++ b/.github/workflows/dep-update.lock.yml @@ -21,7 +21,7 @@ # For more information: https://github.github.com/gh-aw/introduction/overview/ # # -# gh-aw-metadata: {"schema_version":"v2","frontmatter_hash":"09af4968216b71fad2df06de71e1b18285066c897715420667f252576cf84c63","compiler_version":"v0.62.2","strict":true} +# gh-aw-metadata: {"schema_version":"v2","frontmatter_hash":"ee7169a27bb5408e4b1e19ce7105a47c1e38cfeafd703ad9fc69b70cfe607c35","compiler_version":"v0.62.2","strict":true} name: "Update NuGet Dependencies" "on": @@ -1050,7 +1050,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.vsblob.vsassets.io,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.nuget.org,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,azuresearch-usnc.nuget.org,azuresearch-ussc.nuget.org,builds.dotnet.microsoft.com,ci.dot.net,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dc.services.visualstudio.com,dist.nuget.org,dot.net,dotnet.microsoft.com,dotnetcli.blob.core.windows.net,github.com,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,nuget.org,nuget.pkg.github.com,nugetregistryv2prod.blob.core.windows.net,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,oneocsp.microsoft.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,pkgs.dev.azure.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com,www.microsoft.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"create_pull_request\":{\"auto_merge\":true,\"max\":1,\"max_patch_size\":1024,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"AGENTS.md\"],\"protected_path_prefixes\":[\".github/\",\".agents/\"]},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"create_pull_request\":{\"auto_merge\":true,\"max\":1,\"max_patch_size\":1024,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"AGENTS.md\"],\"protected_files_policy\":\"fallback-to-issue\",\"protected_path_prefixes\":[\".github/\",\".agents/\"]},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"}}" GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/dep-update.md b/.github/workflows/dep-update.md index facef23bc..cf574ca4e 100644 --- a/.github/workflows/dep-update.md +++ b/.github/workflows/dep-update.md @@ -21,6 +21,7 @@ tools: safe-outputs: create-pull-request: auto-merge: true + protected-files: fallback-to-issue timeout-minutes: 45 diff --git a/.github/workflows/instruction-drift.agent.lock.yml b/.github/workflows/instruction-drift.agent.lock.yml new file mode 100644 index 000000000..279fad494 --- /dev/null +++ b/.github/workflows/instruction-drift.agent.lock.yml @@ -0,0 +1,1131 @@ +# ___ _ _ +# / _ \ | | (_) +# | |_| | __ _ ___ _ __ | |_ _ ___ +# | _ |/ _` |/ _ \ '_ \| __| |/ __| +# | | | | (_| | __/ | | | |_| | (__ +# \_| |_/\__, |\___|_| |_|\__|_|\___| +# __/ | +# _ _ |___/ +# | | | | / _| | +# | | | | ___ _ __ _ __| |_| | _____ ____ +# | |/\| |/ _ \ '__| |/ /| _| |/ _ \ \ /\ / / ___| +# \ /\ / (_) | | | | ( | | | | (_) \ V V /\__ \ +# \/ \/ \___/|_| |_|\_\|_| |_|\___/ \_/\_/ |___/ +# +# This file was automatically generated by gh-aw (v0.62.2). DO NOT EDIT. +# +# To update this file, edit the corresponding .md file and run: +# gh aw compile +# Not all edits will cause changes to this file. +# +# For more information: https://github.github.com/gh-aw/introduction/overview/ +# +# Weekly check for stale gh-aw instructions. If drift is detected, scans upstream commits and creates a PR updating the skills. +# +# gh-aw-metadata: {"schema_version":"v2","frontmatter_hash":"773f90188be605bb7652acf381e27588ee605e69e3826284603501b6cb91d08b","compiler_version":"v0.62.2","strict":true} + +name: "Instruction Drift Check" +"on": + schedule: + - cron: "45 8 * * 1" + # Friendly format: weekly on monday around 9:00 (scattered) + workflow_dispatch: + +permissions: {} + +concurrency: + cancel-in-progress: false + group: instruction-drift-${{ github.run_id }} + +run-name: "Instruction Drift Check" + +jobs: + activation: + runs-on: ubuntu-slim + permissions: + contents: read + outputs: + comment_id: "" + comment_repo: "" + lockdown_check_failed: ${{ steps.generate_aw_info.outputs.lockdown_check_failed == 'true' }} + model: ${{ steps.generate_aw_info.outputs.model }} + secret_verification_result: ${{ steps.validate-secret.outputs.verification_result }} + steps: + - name: Setup Scripts + uses: github/gh-aw-actions/setup@20045bbd5ad2632b9809856c389708eab1bd16ef # v0.62.2 + with: + destination: ${{ runner.temp }}/gh-aw/actions + - name: Generate agentic run info + id: generate_aw_info + env: + GH_AW_INFO_ENGINE_ID: "copilot" + GH_AW_INFO_ENGINE_NAME: "GitHub Copilot CLI" + GH_AW_INFO_MODEL: "claude-sonnet-4.6" + GH_AW_INFO_VERSION: "" + GH_AW_INFO_AGENT_VERSION: "latest" + GH_AW_INFO_CLI_VERSION: "v0.62.2" + GH_AW_INFO_WORKFLOW_NAME: "Instruction Drift Check" + GH_AW_INFO_EXPERIMENTAL: "false" + GH_AW_INFO_SUPPORTS_TOOLS_ALLOWLIST: "true" + GH_AW_INFO_STAGED: "false" + GH_AW_INFO_ALLOWED_DOMAINS: '["defaults","dotnet"]' + GH_AW_INFO_FIREWALL_ENABLED: "true" + GH_AW_INFO_AWF_VERSION: "v0.24.3" + GH_AW_INFO_AWMG_VERSION: "" + GH_AW_INFO_FIREWALL_TYPE: "squid" + GH_AW_COMPILED_STRICT: "true" + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/generate_aw_info.cjs'); + await main(core, context); + - name: Validate COPILOT_GITHUB_TOKEN secret + id: validate-secret + run: ${RUNNER_TEMP}/gh-aw/actions/validate_multi_secret.sh COPILOT_GITHUB_TOKEN 'GitHub Copilot CLI' https://github.github.com/gh-aw/reference/engines/#github-copilot-default + env: + COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }} + - name: Checkout .github and .agents folders + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + persist-credentials: false + sparse-checkout: | + .github + .agents + sparse-checkout-cone-mode: true + fetch-depth: 1 + - name: Check workflow file timestamps + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_WORKFLOW_FILE: "instruction-drift.agent.lock.yml" + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/check_workflow_timestamp_api.cjs'); + await main(); + - name: Create prompt with built-in context + env: + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + GH_AW_SAFE_OUTPUTS: ${{ env.GH_AW_SAFE_OUTPUTS }} + GH_AW_GITHUB_ACTOR: ${{ github.actor }} + GH_AW_GITHUB_EVENT_COMMENT_ID: ${{ github.event.comment.id }} + GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER: ${{ github.event.discussion.number }} + GH_AW_GITHUB_EVENT_ISSUE_NUMBER: ${{ github.event.issue.number }} + GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER: ${{ github.event.pull_request.number }} + GH_AW_GITHUB_REPOSITORY: ${{ github.repository }} + GH_AW_GITHUB_RUN_ID: ${{ github.run_id }} + GH_AW_GITHUB_WORKSPACE: ${{ github.workspace }} + run: | + bash ${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh + { + cat << 'GH_AW_PROMPT_EOF' + + GH_AW_PROMPT_EOF + cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" + cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" + cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" + cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" + cat << 'GH_AW_PROMPT_EOF' + + Tools: add_comment, create_issue, create_pull_request, missing_tool, missing_data, noop + GH_AW_PROMPT_EOF + cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_create_pull_request.md" + cat << 'GH_AW_PROMPT_EOF' + + + The following GitHub context information is available for this workflow: + {{#if __GH_AW_GITHUB_ACTOR__ }} + - **actor**: __GH_AW_GITHUB_ACTOR__ + {{/if}} + {{#if __GH_AW_GITHUB_REPOSITORY__ }} + - **repository**: __GH_AW_GITHUB_REPOSITORY__ + {{/if}} + {{#if __GH_AW_GITHUB_WORKSPACE__ }} + - **workspace**: __GH_AW_GITHUB_WORKSPACE__ + {{/if}} + {{#if __GH_AW_GITHUB_EVENT_ISSUE_NUMBER__ }} + - **issue-number**: #__GH_AW_GITHUB_EVENT_ISSUE_NUMBER__ + {{/if}} + {{#if __GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER__ }} + - **discussion-number**: #__GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER__ + {{/if}} + {{#if __GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER__ }} + - **pull-request-number**: #__GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER__ + {{/if}} + {{#if __GH_AW_GITHUB_EVENT_COMMENT_ID__ }} + - **comment-id**: __GH_AW_GITHUB_EVENT_COMMENT_ID__ + {{/if}} + {{#if __GH_AW_GITHUB_RUN_ID__ }} + - **workflow-run-id**: __GH_AW_GITHUB_RUN_ID__ + {{/if}} + + + GH_AW_PROMPT_EOF + cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" + cat << 'GH_AW_PROMPT_EOF' + + GH_AW_PROMPT_EOF + cat << 'GH_AW_PROMPT_EOF' + {{#runtime-import .github/workflows/instruction-drift.agent.md}} + GH_AW_PROMPT_EOF + } > "$GH_AW_PROMPT" + - name: Interpolate variables and render templates + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/interpolate_prompt.cjs'); + await main(); + - name: Substitute placeholders + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + GH_AW_GITHUB_ACTOR: ${{ github.actor }} + GH_AW_GITHUB_EVENT_COMMENT_ID: ${{ github.event.comment.id }} + GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER: ${{ github.event.discussion.number }} + GH_AW_GITHUB_EVENT_ISSUE_NUMBER: ${{ github.event.issue.number }} + GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER: ${{ github.event.pull_request.number }} + GH_AW_GITHUB_REPOSITORY: ${{ github.repository }} + GH_AW_GITHUB_RUN_ID: ${{ github.run_id }} + GH_AW_GITHUB_WORKSPACE: ${{ github.workspace }} + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + + const substitutePlaceholders = require('${{ runner.temp }}/gh-aw/actions/substitute_placeholders.cjs'); + + // Call the substitution function + return await substitutePlaceholders({ + file: process.env.GH_AW_PROMPT, + substitutions: { + GH_AW_GITHUB_ACTOR: process.env.GH_AW_GITHUB_ACTOR, + GH_AW_GITHUB_EVENT_COMMENT_ID: process.env.GH_AW_GITHUB_EVENT_COMMENT_ID, + GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER: process.env.GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER, + GH_AW_GITHUB_EVENT_ISSUE_NUMBER: process.env.GH_AW_GITHUB_EVENT_ISSUE_NUMBER, + GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER: process.env.GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER, + GH_AW_GITHUB_REPOSITORY: process.env.GH_AW_GITHUB_REPOSITORY, + GH_AW_GITHUB_RUN_ID: process.env.GH_AW_GITHUB_RUN_ID, + GH_AW_GITHUB_WORKSPACE: process.env.GH_AW_GITHUB_WORKSPACE + } + }); + - name: Validate prompt placeholders + env: + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + run: bash ${RUNNER_TEMP}/gh-aw/actions/validate_prompt_placeholders.sh + - name: Print prompt + env: + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + run: bash ${RUNNER_TEMP}/gh-aw/actions/print_prompt_summary.sh + - name: Upload activation artifact + if: success() + uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0 + with: + name: activation + path: | + /tmp/gh-aw/aw_info.json + /tmp/gh-aw/aw-prompts/prompt.txt + retention-days: 1 + + agent: + needs: activation + runs-on: ubuntu-latest + permissions: + contents: read + pull-requests: read + concurrency: + group: "gh-aw-copilot-${{ github.workflow }}" + env: + DEFAULT_BRANCH: ${{ github.event.repository.default_branch }} + GH_AW_ASSETS_ALLOWED_EXTS: "" + GH_AW_ASSETS_BRANCH: "" + GH_AW_ASSETS_MAX_SIZE_KB: 0 + GH_AW_MCP_LOG_DIR: /tmp/gh-aw/mcp-logs/safeoutputs + GH_AW_WORKFLOW_ID_SANITIZED: instructiondrift.agent + outputs: + checkout_pr_success: ${{ steps.checkout-pr.outputs.checkout_pr_success || 'true' }} + detection_conclusion: ${{ steps.detection_conclusion.outputs.conclusion }} + detection_success: ${{ steps.detection_conclusion.outputs.success }} + has_patch: ${{ steps.collect_output.outputs.has_patch }} + inference_access_error: ${{ steps.detect-inference-error.outputs.inference_access_error || 'false' }} + model: ${{ needs.activation.outputs.model }} + output: ${{ steps.collect_output.outputs.output }} + output_types: ${{ steps.collect_output.outputs.output_types }} + steps: + - name: Setup Scripts + uses: github/gh-aw-actions/setup@20045bbd5ad2632b9809856c389708eab1bd16ef # v0.62.2 + with: + destination: ${{ runner.temp }}/gh-aw/actions + - name: Set runtime paths + run: | + echo "GH_AW_SAFE_OUTPUTS=${RUNNER_TEMP}/gh-aw/safeoutputs/outputs.jsonl" >> "$GITHUB_ENV" + echo "GH_AW_SAFE_OUTPUTS_CONFIG_PATH=${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" >> "$GITHUB_ENV" + echo "GH_AW_SAFE_OUTPUTS_TOOLS_PATH=${RUNNER_TEMP}/gh-aw/safeoutputs/tools.json" >> "$GITHUB_ENV" + - name: Checkout repository + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + persist-credentials: false + - name: Create gh-aw temp directory + run: bash ${RUNNER_TEMP}/gh-aw/actions/create_gh_aw_tmp_dir.sh + - name: Configure gh CLI for GitHub Enterprise + run: bash ${RUNNER_TEMP}/gh-aw/actions/configure_gh_for_ghe.sh + env: + GH_TOKEN: ${{ github.token }} + - name: Configure Git credentials + env: + REPO_NAME: ${{ github.repository }} + SERVER_URL: ${{ github.server_url }} + run: | + git config --global user.email "github-actions[bot]@users.noreply.github.com" + git config --global user.name "github-actions[bot]" + git config --global am.keepcr true + # Re-authenticate git with GitHub token + SERVER_URL_STRIPPED="${SERVER_URL#https://}" + git remote set-url origin "https://x-access-token:${{ github.token }}@${SERVER_URL_STRIPPED}/${REPO_NAME}.git" + echo "Git configured with standard GitHub Actions identity" + - name: Checkout PR branch + id: checkout-pr + if: | + (github.event.pull_request) || (github.event.issue.pull_request) + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + with: + github-token: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/checkout_pr_branch.cjs'); + await main(); + - name: Install GitHub Copilot CLI + run: ${RUNNER_TEMP}/gh-aw/actions/install_copilot_cli.sh latest + env: + GH_HOST: github.com + - name: Install AWF binary + run: bash ${RUNNER_TEMP}/gh-aw/actions/install_awf_binary.sh v0.24.3 + - name: Determine automatic lockdown mode for GitHub MCP Server + id: determine-automatic-lockdown + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_GITHUB_TOKEN: ${{ secrets.GH_AW_GITHUB_TOKEN }} + GH_AW_GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN }} + with: + script: | + const determineAutomaticLockdown = require('${{ runner.temp }}/gh-aw/actions/determine_automatic_lockdown.cjs'); + await determineAutomaticLockdown(github, context, core); + - name: Download container images + run: bash ${RUNNER_TEMP}/gh-aw/actions/download_docker_images.sh ghcr.io/github/gh-aw-firewall/agent:0.24.3 ghcr.io/github/gh-aw-firewall/api-proxy:0.24.3 ghcr.io/github/gh-aw-firewall/squid:0.24.3 ghcr.io/github/gh-aw-mcpg:v0.1.19 ghcr.io/github/github-mcp-server:v0.32.0 node:lts-alpine + - name: Write Safe Outputs Config + run: | + mkdir -p ${RUNNER_TEMP}/gh-aw/safeoutputs + mkdir -p /tmp/gh-aw/safeoutputs + mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs + cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/config.json << 'GH_AW_SAFE_OUTPUTS_CONFIG_EOF' + {"add_comment":{"max":1},"create_issue":{"expires":720,"max":1},"create_pull_request":{"draft":true,"expires":336,"max":1,"title_prefix":"[drift] "},"missing_data":{},"missing_tool":{},"noop":{"max":1}} + GH_AW_SAFE_OUTPUTS_CONFIG_EOF + - name: Write Safe Outputs Tools + run: | + cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/tools_meta.json << 'GH_AW_SAFE_OUTPUTS_TOOLS_META_EOF' + { + "description_suffixes": { + "add_comment": " CONSTRAINTS: Maximum 1 comment(s) can be added.", + "create_issue": " CONSTRAINTS: Maximum 1 issue(s) can be created. Title will be prefixed with \"[drift] \". Labels [\"instruction-drift\"] will be automatically added.", + "create_pull_request": " CONSTRAINTS: Maximum 1 pull request(s) can be created. Title will be prefixed with \"[drift] \". Labels [\"instruction-drift\" \"automation\"] will be automatically added. PRs will be created as drafts." + }, + "repo_params": {}, + "dynamic_tools": [] + } + GH_AW_SAFE_OUTPUTS_TOOLS_META_EOF + cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/validation.json << 'GH_AW_SAFE_OUTPUTS_VALIDATION_EOF' + { + "add_comment": { + "defaultMax": 1, + "fields": { + "body": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "item_number": { + "issueOrPRNumber": true + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, + "create_issue": { + "defaultMax": 1, + "fields": { + "body": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "labels": { + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "parent": { + "issueOrPRNumber": true + }, + "repo": { + "type": "string", + "maxLength": 256 + }, + "temporary_id": { + "type": "string" + }, + "title": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 128 + } + } + }, + "create_pull_request": { + "defaultMax": 1, + "fields": { + "body": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "branch": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 256 + }, + "draft": { + "type": "boolean" + }, + "labels": { + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "repo": { + "type": "string", + "maxLength": 256 + }, + "title": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 128 + } + } + }, + "missing_data": { + "defaultMax": 20, + "fields": { + "alternatives": { + "type": "string", + "sanitize": true, + "maxLength": 256 + }, + "context": { + "type": "string", + "sanitize": true, + "maxLength": 256 + }, + "data_type": { + "type": "string", + "sanitize": true, + "maxLength": 128 + }, + "reason": { + "type": "string", + "sanitize": true, + "maxLength": 256 + } + } + }, + "missing_tool": { + "defaultMax": 20, + "fields": { + "alternatives": { + "type": "string", + "sanitize": true, + "maxLength": 512 + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 256 + }, + "tool": { + "type": "string", + "sanitize": true, + "maxLength": 128 + } + } + }, + "noop": { + "defaultMax": 1, + "fields": { + "message": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + } + } + } + } + GH_AW_SAFE_OUTPUTS_VALIDATION_EOF + node ${RUNNER_TEMP}/gh-aw/actions/generate_safe_outputs_tools.cjs + - name: Generate Safe Outputs MCP Server Config + id: safe-outputs-config + run: | + # Generate a secure random API key (360 bits of entropy, 40+ chars) + # Mask immediately to prevent timing vulnerabilities + API_KEY=$(openssl rand -base64 45 | tr -d '/+=') + echo "::add-mask::${API_KEY}" + + PORT=3001 + + # Set outputs for next steps + { + echo "safe_outputs_api_key=${API_KEY}" + echo "safe_outputs_port=${PORT}" + } >> "$GITHUB_OUTPUT" + + echo "Safe Outputs MCP server will run on port ${PORT}" + + - name: Start Safe Outputs MCP HTTP Server + id: safe-outputs-start + env: + DEBUG: '*' + GH_AW_SAFE_OUTPUTS_PORT: ${{ steps.safe-outputs-config.outputs.safe_outputs_port }} + GH_AW_SAFE_OUTPUTS_API_KEY: ${{ steps.safe-outputs-config.outputs.safe_outputs_api_key }} + GH_AW_SAFE_OUTPUTS_TOOLS_PATH: ${{ runner.temp }}/gh-aw/safeoutputs/tools.json + GH_AW_SAFE_OUTPUTS_CONFIG_PATH: ${{ runner.temp }}/gh-aw/safeoutputs/config.json + GH_AW_MCP_LOG_DIR: /tmp/gh-aw/mcp-logs/safeoutputs + run: | + # Environment variables are set above to prevent template injection + export DEBUG + export GH_AW_SAFE_OUTPUTS_PORT + export GH_AW_SAFE_OUTPUTS_API_KEY + export GH_AW_SAFE_OUTPUTS_TOOLS_PATH + export GH_AW_SAFE_OUTPUTS_CONFIG_PATH + export GH_AW_MCP_LOG_DIR + + bash ${RUNNER_TEMP}/gh-aw/actions/start_safe_outputs_server.sh + + - name: Start MCP Gateway + id: start-mcp-gateway + env: + GH_AW_SAFE_OUTPUTS: ${{ env.GH_AW_SAFE_OUTPUTS }} + GH_AW_SAFE_OUTPUTS_API_KEY: ${{ steps.safe-outputs-start.outputs.api_key }} + GH_AW_SAFE_OUTPUTS_PORT: ${{ steps.safe-outputs-start.outputs.port }} + GITHUB_MCP_GUARD_MIN_INTEGRITY: ${{ steps.determine-automatic-lockdown.outputs.min_integrity }} + GITHUB_MCP_GUARD_REPOS: ${{ steps.determine-automatic-lockdown.outputs.repos }} + GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + run: | + set -eo pipefail + mkdir -p /tmp/gh-aw/mcp-config + + # Export gateway environment variables for MCP config and gateway script + export MCP_GATEWAY_PORT="80" + export MCP_GATEWAY_DOMAIN="host.docker.internal" + MCP_GATEWAY_API_KEY=$(openssl rand -base64 45 | tr -d '/+=') + echo "::add-mask::${MCP_GATEWAY_API_KEY}" + export MCP_GATEWAY_API_KEY + export MCP_GATEWAY_PAYLOAD_DIR="/tmp/gh-aw/mcp-payloads" + mkdir -p "${MCP_GATEWAY_PAYLOAD_DIR}" + export MCP_GATEWAY_PAYLOAD_SIZE_THRESHOLD="524288" + export DEBUG="*" + + export GH_AW_ENGINE="copilot" + export MCP_GATEWAY_DOCKER_COMMAND='docker run -i --rm --network host -v /var/run/docker.sock:/var/run/docker.sock -e MCP_GATEWAY_PORT -e MCP_GATEWAY_DOMAIN -e MCP_GATEWAY_API_KEY -e MCP_GATEWAY_PAYLOAD_DIR -e MCP_GATEWAY_PAYLOAD_SIZE_THRESHOLD -e DEBUG -e MCP_GATEWAY_LOG_DIR -e GH_AW_MCP_LOG_DIR -e GH_AW_SAFE_OUTPUTS -e GH_AW_SAFE_OUTPUTS_CONFIG_PATH -e GH_AW_SAFE_OUTPUTS_TOOLS_PATH -e GH_AW_ASSETS_BRANCH -e GH_AW_ASSETS_MAX_SIZE_KB -e GH_AW_ASSETS_ALLOWED_EXTS -e DEFAULT_BRANCH -e GITHUB_MCP_SERVER_TOKEN -e GITHUB_MCP_GUARD_MIN_INTEGRITY -e GITHUB_MCP_GUARD_REPOS -e GITHUB_REPOSITORY -e GITHUB_SERVER_URL -e GITHUB_SHA -e GITHUB_WORKSPACE -e GITHUB_TOKEN -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e GITHUB_JOB -e GITHUB_ACTION -e GITHUB_EVENT_NAME -e GITHUB_EVENT_PATH -e GITHUB_ACTOR -e GITHUB_ACTOR_ID -e GITHUB_TRIGGERING_ACTOR -e GITHUB_WORKFLOW -e GITHUB_WORKFLOW_REF -e GITHUB_WORKFLOW_SHA -e GITHUB_REF -e GITHUB_REF_NAME -e GITHUB_REF_TYPE -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GH_AW_SAFE_OUTPUTS_PORT -e GH_AW_SAFE_OUTPUTS_API_KEY -v /tmp/gh-aw/mcp-payloads:/tmp/gh-aw/mcp-payloads:rw -v /opt:/opt:ro -v /tmp:/tmp:rw -v '"${GITHUB_WORKSPACE}"':'"${GITHUB_WORKSPACE}"':rw ghcr.io/github/gh-aw-mcpg:v0.1.19' + + mkdir -p /home/runner/.copilot + cat << GH_AW_MCP_CONFIG_EOF | bash ${RUNNER_TEMP}/gh-aw/actions/start_mcp_gateway.sh + { + "mcpServers": { + "github": { + "type": "stdio", + "container": "ghcr.io/github/github-mcp-server:v0.32.0", + "env": { + "GITHUB_HOST": "\${GITHUB_SERVER_URL}", + "GITHUB_PERSONAL_ACCESS_TOKEN": "\${GITHUB_MCP_SERVER_TOKEN}", + "GITHUB_READ_ONLY": "1", + "GITHUB_TOOLSETS": "repos,pull_requests" + }, + "guard-policies": { + "allow-only": { + "min-integrity": "$GITHUB_MCP_GUARD_MIN_INTEGRITY", + "repos": "$GITHUB_MCP_GUARD_REPOS" + } + } + }, + "safeoutputs": { + "type": "http", + "url": "http://host.docker.internal:$GH_AW_SAFE_OUTPUTS_PORT", + "headers": { + "Authorization": "\${GH_AW_SAFE_OUTPUTS_API_KEY}" + }, + "guard-policies": { + "write-sink": { + "accept": [ + "*" + ] + } + } + } + }, + "gateway": { + "port": $MCP_GATEWAY_PORT, + "domain": "${MCP_GATEWAY_DOMAIN}", + "apiKey": "${MCP_GATEWAY_API_KEY}", + "payloadDir": "${MCP_GATEWAY_PAYLOAD_DIR}" + } + } + GH_AW_MCP_CONFIG_EOF + - name: Download activation artifact + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: activation + path: /tmp/gh-aw + - name: Clean git credentials + continue-on-error: true + run: bash ${RUNNER_TEMP}/gh-aw/actions/clean_git_credentials.sh + - name: Execute GitHub Copilot CLI + id: agentic_execution + # Copilot CLI tool arguments (sorted): + timeout-minutes: 30 + run: | + set -o pipefail + touch /tmp/gh-aw/agent-step-summary.md + # shellcheck disable=SC1003 + sudo -E awf --env-all --container-workdir "${GITHUB_WORKSPACE}" --mount "${RUNNER_TEMP}/gh-aw:${RUNNER_TEMP}/gh-aw:ro" --mount "${RUNNER_TEMP}/gh-aw:/host${RUNNER_TEMP}/gh-aw:ro" --allow-domains "*.vsblob.vsassets.io,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.nuget.org,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,azuresearch-usnc.nuget.org,azuresearch-ussc.nuget.org,builds.dotnet.microsoft.com,ci.dot.net,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dc.services.visualstudio.com,dist.nuget.org,dot.net,dotnet.microsoft.com,dotnetcli.blob.core.windows.net,github.com,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,nuget.org,nuget.pkg.github.com,nugetregistryv2prod.blob.core.windows.net,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,oneocsp.microsoft.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,pkgs.dev.azure.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com,www.microsoft.com" --log-level info --proxy-logs-dir /tmp/gh-aw/sandbox/firewall/logs --enable-host-access --image-tag 0.24.3 --skip-pull --enable-api-proxy \ + -- /bin/bash -c '/usr/local/bin/copilot --add-dir /tmp/gh-aw/ --log-level all --log-dir /tmp/gh-aw/sandbox/agent/logs/ --add-dir "${GITHUB_WORKSPACE}" --disable-builtin-mcps --allow-all-tools --allow-all-paths --prompt "$(cat /tmp/gh-aw/aw-prompts/prompt.txt)"' 2>&1 | tee -a /tmp/gh-aw/agent-stdio.log + env: + COPILOT_AGENT_RUNNER_TYPE: STANDALONE + COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }} + COPILOT_MODEL: claude-sonnet-4.6 + GH_AW_MCP_CONFIG: /home/runner/.copilot/mcp-config.json + GH_AW_PHASE: agent + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + GH_AW_SAFE_OUTPUTS: ${{ env.GH_AW_SAFE_OUTPUTS }} + GH_AW_VERSION: v0.62.2 + GITHUB_API_URL: ${{ github.api_url }} + GITHUB_AW: true + GITHUB_HEAD_REF: ${{ github.head_ref }} + GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + GITHUB_REF_NAME: ${{ github.ref_name }} + GITHUB_SERVER_URL: ${{ github.server_url }} + GITHUB_STEP_SUMMARY: /tmp/gh-aw/agent-step-summary.md + GITHUB_WORKSPACE: ${{ github.workspace }} + GIT_AUTHOR_EMAIL: github-actions[bot]@users.noreply.github.com + GIT_AUTHOR_NAME: github-actions[bot] + GIT_COMMITTER_EMAIL: github-actions[bot]@users.noreply.github.com + GIT_COMMITTER_NAME: github-actions[bot] + XDG_CONFIG_HOME: /home/runner + - name: Detect inference access error + id: detect-inference-error + if: always() + continue-on-error: true + run: bash ${RUNNER_TEMP}/gh-aw/actions/detect_inference_access_error.sh + - name: Configure Git credentials + env: + REPO_NAME: ${{ github.repository }} + SERVER_URL: ${{ github.server_url }} + run: | + git config --global user.email "github-actions[bot]@users.noreply.github.com" + git config --global user.name "github-actions[bot]" + git config --global am.keepcr true + # Re-authenticate git with GitHub token + SERVER_URL_STRIPPED="${SERVER_URL#https://}" + git remote set-url origin "https://x-access-token:${{ github.token }}@${SERVER_URL_STRIPPED}/${REPO_NAME}.git" + echo "Git configured with standard GitHub Actions identity" + - name: Copy Copilot session state files to logs + if: always() + continue-on-error: true + run: | + # Copy Copilot session state files to logs folder for artifact collection + # This ensures they are in /tmp/gh-aw/ where secret redaction can scan them + SESSION_STATE_DIR="$HOME/.copilot/session-state" + LOGS_DIR="/tmp/gh-aw/sandbox/agent/logs" + + if [ -d "$SESSION_STATE_DIR" ]; then + echo "Copying Copilot session state files from $SESSION_STATE_DIR to $LOGS_DIR" + mkdir -p "$LOGS_DIR" + cp -v "$SESSION_STATE_DIR"/*.jsonl "$LOGS_DIR/" 2>/dev/null || true + echo "Session state files copied successfully" + else + echo "No session-state directory found at $SESSION_STATE_DIR" + fi + - name: Stop MCP Gateway + if: always() + continue-on-error: true + env: + MCP_GATEWAY_PORT: ${{ steps.start-mcp-gateway.outputs.gateway-port }} + MCP_GATEWAY_API_KEY: ${{ steps.start-mcp-gateway.outputs.gateway-api-key }} + GATEWAY_PID: ${{ steps.start-mcp-gateway.outputs.gateway-pid }} + run: | + bash ${RUNNER_TEMP}/gh-aw/actions/stop_mcp_gateway.sh "$GATEWAY_PID" + - name: Redact secrets in logs + if: always() + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/redact_secrets.cjs'); + await main(); + env: + GH_AW_SECRET_NAMES: 'COPILOT_GITHUB_TOKEN,GH_AW_GITHUB_MCP_SERVER_TOKEN,GH_AW_GITHUB_TOKEN,GITHUB_TOKEN' + SECRET_COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }} + SECRET_GH_AW_GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN }} + SECRET_GH_AW_GITHUB_TOKEN: ${{ secrets.GH_AW_GITHUB_TOKEN }} + SECRET_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + - name: Append agent step summary + if: always() + run: bash ${RUNNER_TEMP}/gh-aw/actions/append_agent_step_summary.sh + - name: Copy Safe Outputs + if: always() + run: | + mkdir -p /tmp/gh-aw + cp "$GH_AW_SAFE_OUTPUTS" /tmp/gh-aw/safeoutputs.jsonl 2>/dev/null || true + - name: Ingest agent output + id: collect_output + if: always() + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_SAFE_OUTPUTS: ${{ env.GH_AW_SAFE_OUTPUTS }} + GH_AW_ALLOWED_DOMAINS: "*.vsblob.vsassets.io,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.nuget.org,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,azuresearch-usnc.nuget.org,azuresearch-ussc.nuget.org,builds.dotnet.microsoft.com,ci.dot.net,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dc.services.visualstudio.com,dist.nuget.org,dot.net,dotnet.microsoft.com,dotnetcli.blob.core.windows.net,github.com,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,nuget.org,nuget.pkg.github.com,nugetregistryv2prod.blob.core.windows.net,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,oneocsp.microsoft.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,pkgs.dev.azure.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com,www.microsoft.com" + GITHUB_SERVER_URL: ${{ github.server_url }} + GITHUB_API_URL: ${{ github.api_url }} + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/collect_ndjson_output.cjs'); + await main(); + - name: Parse agent logs for step summary + if: always() + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: /tmp/gh-aw/sandbox/agent/logs/ + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/parse_copilot_log.cjs'); + await main(); + - name: Parse MCP Gateway logs for step summary + if: always() + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/parse_mcp_gateway_log.cjs'); + await main(); + - name: Print firewall logs + if: always() + continue-on-error: true + env: + AWF_LOGS_DIR: /tmp/gh-aw/sandbox/firewall/logs + run: | + # Fix permissions on firewall logs so they can be uploaded as artifacts + # AWF runs with sudo, creating files owned by root + sudo chmod -R a+r /tmp/gh-aw/sandbox/firewall/logs 2>/dev/null || true + # Only run awf logs summary if awf command exists (it may not be installed if workflow failed before install step) + if command -v awf &> /dev/null; then + awf logs summary | tee -a "$GITHUB_STEP_SUMMARY" + else + echo 'AWF binary not installed, skipping firewall log summary' + fi + - name: Upload agent artifacts + if: always() + continue-on-error: true + uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0 + with: + name: agent + path: | + /tmp/gh-aw/aw-prompts/prompt.txt + /tmp/gh-aw/sandbox/agent/logs/ + /tmp/gh-aw/redacted-urls.log + /tmp/gh-aw/mcp-logs/ + /tmp/gh-aw/sandbox/firewall/logs/ + /tmp/gh-aw/agent-stdio.log + /tmp/gh-aw/agent/ + /tmp/gh-aw/safeoutputs.jsonl + /tmp/gh-aw/agent_output.json + /tmp/gh-aw/aw-*.patch + if-no-files-found: ignore + # --- Threat Detection (inline) --- + - name: Check if detection needed + id: detection_guard + if: always() + env: + OUTPUT_TYPES: ${{ steps.collect_output.outputs.output_types }} + HAS_PATCH: ${{ steps.collect_output.outputs.has_patch }} + run: | + if [[ -n "$OUTPUT_TYPES" || "$HAS_PATCH" == "true" ]]; then + echo "run_detection=true" >> "$GITHUB_OUTPUT" + echo "Detection will run: output_types=$OUTPUT_TYPES, has_patch=$HAS_PATCH" + else + echo "run_detection=false" >> "$GITHUB_OUTPUT" + echo "Detection skipped: no agent outputs or patches to analyze" + fi + - name: Clear MCP configuration for detection + if: always() && steps.detection_guard.outputs.run_detection == 'true' + run: | + rm -f /tmp/gh-aw/mcp-config/mcp-servers.json + rm -f /home/runner/.copilot/mcp-config.json + rm -f "$GITHUB_WORKSPACE/.gemini/settings.json" + - name: Prepare threat detection files + if: always() && steps.detection_guard.outputs.run_detection == 'true' + run: | + mkdir -p /tmp/gh-aw/threat-detection/aw-prompts + cp /tmp/gh-aw/aw-prompts/prompt.txt /tmp/gh-aw/threat-detection/aw-prompts/prompt.txt 2>/dev/null || true + cp /tmp/gh-aw/agent_output.json /tmp/gh-aw/threat-detection/agent_output.json 2>/dev/null || true + for f in /tmp/gh-aw/aw-*.patch; do + [ -f "$f" ] && cp "$f" /tmp/gh-aw/threat-detection/ 2>/dev/null || true + done + echo "Prepared threat detection files:" + ls -la /tmp/gh-aw/threat-detection/ 2>/dev/null || true + - name: Setup threat detection + if: always() && steps.detection_guard.outputs.run_detection == 'true' + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + WORKFLOW_NAME: "Instruction Drift Check" + WORKFLOW_DESCRIPTION: "Weekly check for stale gh-aw instructions. If drift is detected, scans upstream commits and creates a PR updating the skills." + HAS_PATCH: ${{ steps.collect_output.outputs.has_patch }} + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/setup_threat_detection.cjs'); + await main(); + - name: Ensure threat-detection directory and log + if: always() && steps.detection_guard.outputs.run_detection == 'true' + run: | + mkdir -p /tmp/gh-aw/threat-detection + touch /tmp/gh-aw/threat-detection/detection.log + - name: Execute GitHub Copilot CLI + if: always() && steps.detection_guard.outputs.run_detection == 'true' + id: detection_agentic_execution + # Copilot CLI tool arguments (sorted): + # --allow-tool shell(cat) + # --allow-tool shell(grep) + # --allow-tool shell(head) + # --allow-tool shell(jq) + # --allow-tool shell(ls) + # --allow-tool shell(tail) + # --allow-tool shell(wc) + timeout-minutes: 20 + run: | + set -o pipefail + touch /tmp/gh-aw/agent-step-summary.md + # shellcheck disable=SC1003 + sudo -E awf --env-all --container-workdir "${GITHUB_WORKSPACE}" --mount "${RUNNER_TEMP}/gh-aw:${RUNNER_TEMP}/gh-aw:ro" --mount "${RUNNER_TEMP}/gh-aw:/host${RUNNER_TEMP}/gh-aw:ro" --allow-domains "api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,github.com,host.docker.internal,raw.githubusercontent.com,registry.npmjs.org,telemetry.enterprise.githubcopilot.com" --log-level info --proxy-logs-dir /tmp/gh-aw/sandbox/firewall/logs --enable-host-access --image-tag 0.24.3 --skip-pull --enable-api-proxy \ + -- /bin/bash -c '/usr/local/bin/copilot --add-dir /tmp/gh-aw/ --log-level all --log-dir /tmp/gh-aw/sandbox/agent/logs/ --add-dir "${GITHUB_WORKSPACE}" --disable-builtin-mcps --allow-tool '\''shell(cat)'\'' --allow-tool '\''shell(grep)'\'' --allow-tool '\''shell(head)'\'' --allow-tool '\''shell(jq)'\'' --allow-tool '\''shell(ls)'\'' --allow-tool '\''shell(tail)'\'' --allow-tool '\''shell(wc)'\'' --prompt "$(cat /tmp/gh-aw/aw-prompts/prompt.txt)"' 2>&1 | tee -a /tmp/gh-aw/threat-detection/detection.log + env: + COPILOT_AGENT_RUNNER_TYPE: STANDALONE + COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }} + COPILOT_MODEL: claude-sonnet-4.6 + GH_AW_PHASE: detection + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + GH_AW_VERSION: v0.62.2 + GITHUB_API_URL: ${{ github.api_url }} + GITHUB_AW: true + GITHUB_HEAD_REF: ${{ github.head_ref }} + GITHUB_REF_NAME: ${{ github.ref_name }} + GITHUB_SERVER_URL: ${{ github.server_url }} + GITHUB_STEP_SUMMARY: /tmp/gh-aw/agent-step-summary.md + GITHUB_WORKSPACE: ${{ github.workspace }} + GIT_AUTHOR_EMAIL: github-actions[bot]@users.noreply.github.com + GIT_AUTHOR_NAME: github-actions[bot] + GIT_COMMITTER_EMAIL: github-actions[bot]@users.noreply.github.com + GIT_COMMITTER_NAME: github-actions[bot] + XDG_CONFIG_HOME: /home/runner + - name: Parse threat detection results + id: parse_detection_results + if: always() && steps.detection_guard.outputs.run_detection == 'true' + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/parse_threat_detection_results.cjs'); + await main(); + - name: Upload threat detection log + if: always() && steps.detection_guard.outputs.run_detection == 'true' + uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0 + with: + name: detection + path: /tmp/gh-aw/threat-detection/detection.log + if-no-files-found: ignore + - name: Set detection conclusion + id: detection_conclusion + if: always() + env: + RUN_DETECTION: ${{ steps.detection_guard.outputs.run_detection }} + DETECTION_SUCCESS: ${{ steps.parse_detection_results.outputs.success }} + run: | + if [[ "$RUN_DETECTION" != "true" ]]; then + echo "conclusion=skipped" >> "$GITHUB_OUTPUT" + echo "success=true" >> "$GITHUB_OUTPUT" + echo "Detection was not needed, marking as skipped" + elif [[ "$DETECTION_SUCCESS" == "true" ]]; then + echo "conclusion=success" >> "$GITHUB_OUTPUT" + echo "success=true" >> "$GITHUB_OUTPUT" + echo "Detection passed successfully" + else + echo "conclusion=failure" >> "$GITHUB_OUTPUT" + echo "success=false" >> "$GITHUB_OUTPUT" + echo "Detection found issues" + fi + + conclusion: + needs: + - activation + - agent + - safe_outputs + if: (always()) && ((needs.agent.result != 'skipped') || (needs.activation.outputs.lockdown_check_failed == 'true')) + runs-on: ubuntu-slim + permissions: + contents: write + discussions: write + issues: write + pull-requests: write + concurrency: + group: "gh-aw-conclusion-instruction-drift.agent" + cancel-in-progress: false + outputs: + noop_message: ${{ steps.noop.outputs.noop_message }} + tools_reported: ${{ steps.missing_tool.outputs.tools_reported }} + total_count: ${{ steps.missing_tool.outputs.total_count }} + steps: + - name: Setup Scripts + uses: github/gh-aw-actions/setup@20045bbd5ad2632b9809856c389708eab1bd16ef # v0.62.2 + with: + destination: ${{ runner.temp }}/gh-aw/actions + - name: Download agent output artifact + id: download-agent-output + continue-on-error: true + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: agent + path: /tmp/gh-aw/ + - name: Setup agent output environment variable + if: steps.download-agent-output.outcome == 'success' + run: | + mkdir -p /tmp/gh-aw/ + find "/tmp/gh-aw/" -type f -print + echo "GH_AW_AGENT_OUTPUT=/tmp/gh-aw/agent_output.json" >> "$GITHUB_ENV" + - name: Process No-Op Messages + id: noop + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: ${{ env.GH_AW_AGENT_OUTPUT }} + GH_AW_NOOP_MAX: "1" + GH_AW_WORKFLOW_NAME: "Instruction Drift Check" + with: + github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/noop.cjs'); + await main(); + - name: Record Missing Tool + id: missing_tool + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: ${{ env.GH_AW_AGENT_OUTPUT }} + GH_AW_WORKFLOW_NAME: "Instruction Drift Check" + with: + github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/missing_tool.cjs'); + await main(); + - name: Handle Agent Failure + id: handle_agent_failure + if: always() + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: ${{ env.GH_AW_AGENT_OUTPUT }} + GH_AW_WORKFLOW_NAME: "Instruction Drift Check" + GH_AW_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + GH_AW_AGENT_CONCLUSION: ${{ needs.agent.result }} + GH_AW_WORKFLOW_ID: "instruction-drift.agent" + GH_AW_SECRET_VERIFICATION_RESULT: ${{ needs.activation.outputs.secret_verification_result }} + GH_AW_CHECKOUT_PR_SUCCESS: ${{ needs.agent.outputs.checkout_pr_success }} + GH_AW_INFERENCE_ACCESS_ERROR: ${{ needs.agent.outputs.inference_access_error }} + GH_AW_CODE_PUSH_FAILURE_ERRORS: ${{ needs.safe_outputs.outputs.code_push_failure_errors }} + GH_AW_CODE_PUSH_FAILURE_COUNT: ${{ needs.safe_outputs.outputs.code_push_failure_count }} + GH_AW_LOCKDOWN_CHECK_FAILED: ${{ needs.activation.outputs.lockdown_check_failed }} + GH_AW_GROUP_REPORTS: "false" + GH_AW_FAILURE_REPORT_AS_ISSUE: "true" + GH_AW_TIMEOUT_MINUTES: "30" + with: + github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/handle_agent_failure.cjs'); + await main(); + - name: Handle No-Op Message + id: handle_noop_message + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: ${{ env.GH_AW_AGENT_OUTPUT }} + GH_AW_WORKFLOW_NAME: "Instruction Drift Check" + GH_AW_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + GH_AW_AGENT_CONCLUSION: ${{ needs.agent.result }} + GH_AW_NOOP_MESSAGE: ${{ steps.noop.outputs.noop_message }} + GH_AW_NOOP_REPORT_AS_ISSUE: "true" + with: + github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/handle_noop_message.cjs'); + await main(); + - name: Handle Create Pull Request Error + id: handle_create_pr_error + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: ${{ env.GH_AW_AGENT_OUTPUT }} + GH_AW_WORKFLOW_NAME: "Instruction Drift Check" + GH_AW_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + with: + github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/handle_create_pr_error.cjs'); + await main(); + + safe_outputs: + needs: + - activation + - agent + if: ((!cancelled()) && (needs.agent.result != 'skipped')) && (needs.agent.outputs.detection_success == 'true') + runs-on: ubuntu-slim + permissions: + contents: write + discussions: write + issues: write + pull-requests: write + timeout-minutes: 15 + env: + GH_AW_CALLER_WORKFLOW_ID: "${{ github.repository }}/instruction-drift.agent" + GH_AW_ENGINE_ID: "copilot" + GH_AW_ENGINE_MODEL: "claude-sonnet-4.6" + GH_AW_WORKFLOW_ID: "instruction-drift.agent" + GH_AW_WORKFLOW_NAME: "Instruction Drift Check" + outputs: + code_push_failure_count: ${{ steps.process_safe_outputs.outputs.code_push_failure_count }} + code_push_failure_errors: ${{ steps.process_safe_outputs.outputs.code_push_failure_errors }} + comment_id: ${{ steps.process_safe_outputs.outputs.comment_id }} + comment_url: ${{ steps.process_safe_outputs.outputs.comment_url }} + create_discussion_error_count: ${{ steps.process_safe_outputs.outputs.create_discussion_error_count }} + create_discussion_errors: ${{ steps.process_safe_outputs.outputs.create_discussion_errors }} + created_issue_number: ${{ steps.process_safe_outputs.outputs.created_issue_number }} + created_issue_url: ${{ steps.process_safe_outputs.outputs.created_issue_url }} + created_pr_number: ${{ steps.process_safe_outputs.outputs.created_pr_number }} + created_pr_url: ${{ steps.process_safe_outputs.outputs.created_pr_url }} + process_safe_outputs_processed_count: ${{ steps.process_safe_outputs.outputs.processed_count }} + process_safe_outputs_temporary_id_map: ${{ steps.process_safe_outputs.outputs.temporary_id_map }} + steps: + - name: Setup Scripts + uses: github/gh-aw-actions/setup@20045bbd5ad2632b9809856c389708eab1bd16ef # v0.62.2 + with: + destination: ${{ runner.temp }}/gh-aw/actions + - name: Download agent output artifact + id: download-agent-output + continue-on-error: true + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: agent + path: /tmp/gh-aw/ + - name: Setup agent output environment variable + if: steps.download-agent-output.outcome == 'success' + run: | + mkdir -p /tmp/gh-aw/ + find "/tmp/gh-aw/" -type f -print + echo "GH_AW_AGENT_OUTPUT=/tmp/gh-aw/agent_output.json" >> "$GITHUB_ENV" + - name: Download patch artifact + continue-on-error: true + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: agent + path: /tmp/gh-aw/ + - name: Checkout repository + if: ((!cancelled()) && (needs.agent.result != 'skipped')) && (contains(needs.agent.outputs.output_types, 'create_pull_request')) + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + ref: ${{ github.base_ref || github.event.pull_request.base.ref || github.ref_name || github.event.repository.default_branch }} + token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + persist-credentials: false + fetch-depth: 1 + - name: Configure Git credentials + if: ((!cancelled()) && (needs.agent.result != 'skipped')) && (contains(needs.agent.outputs.output_types, 'create_pull_request')) + env: + REPO_NAME: ${{ github.repository }} + SERVER_URL: ${{ github.server_url }} + GIT_TOKEN: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + run: | + git config --global user.email "github-actions[bot]@users.noreply.github.com" + git config --global user.name "github-actions[bot]" + git config --global am.keepcr true + # Re-authenticate git with GitHub token + SERVER_URL_STRIPPED="${SERVER_URL#https://}" + git remote set-url origin "https://x-access-token:${GIT_TOKEN}@${SERVER_URL_STRIPPED}/${REPO_NAME}.git" + echo "Git configured with standard GitHub Actions identity" + - name: Configure GH_HOST for enterprise compatibility + shell: bash + run: | + # Derive GH_HOST from GITHUB_SERVER_URL so the gh CLI targets the correct + # GitHub instance (GHES/GHEC). On github.com this is a harmless no-op. + GH_HOST="${GITHUB_SERVER_URL#https://}" + GH_HOST="${GH_HOST#http://}" + echo "GH_HOST=${GH_HOST}" >> "$GITHUB_ENV" + - name: Process Safe Outputs + id: process_safe_outputs + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: ${{ env.GH_AW_AGENT_OUTPUT }} + GH_AW_ALLOWED_DOMAINS: "*.vsblob.vsassets.io,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.nuget.org,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,azuresearch-usnc.nuget.org,azuresearch-ussc.nuget.org,builds.dotnet.microsoft.com,ci.dot.net,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dc.services.visualstudio.com,dist.nuget.org,dot.net,dotnet.microsoft.com,dotnetcli.blob.core.windows.net,github.com,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,nuget.org,nuget.pkg.github.com,nugetregistryv2prod.blob.core.windows.net,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,oneocsp.microsoft.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,pkgs.dev.azure.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com,www.microsoft.com" + GITHUB_SERVER_URL: ${{ github.server_url }} + GITHUB_API_URL: ${{ github.api_url }} + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"max\":1},\"create_issue\":{\"close_older_issues\":true,\"expires\":720,\"labels\":[\"instruction-drift\"],\"max\":1,\"title_prefix\":\"[drift] \"},\"create_pull_request\":{\"draft\":true,\"expires\":336,\"labels\":[\"instruction-drift\",\"automation\"],\"max\":1,\"max_patch_size\":1024,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"AGENTS.md\"],\"protected_files_policy\":\"allowed\",\"protected_path_prefixes\":[\".github/\",\".agents/\"],\"title_prefix\":\"[drift] \"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"}}" + GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} + with: + github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('${{ runner.temp }}/gh-aw/actions/safe_output_handler_manager.cjs'); + await main(); + - name: Upload safe output items + if: always() + uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0 + with: + name: safe-output-items + path: /tmp/gh-aw/safe-output-items.jsonl + if-no-files-found: ignore + diff --git a/.github/workflows/instruction-drift.agent.md b/.github/workflows/instruction-drift.agent.md new file mode 100644 index 000000000..6b9cbd74a --- /dev/null +++ b/.github/workflows/instruction-drift.agent.md @@ -0,0 +1,156 @@ +--- +name: "Instruction Drift Check" +description: "Weekly check for stale gh-aw instructions. If drift is detected, scans upstream commits and creates a PR updating the skills." + +on: + schedule: weekly on monday around 9:00 + workflow_dispatch: + +permissions: + contents: read + pull-requests: read + +engine: + id: copilot + model: claude-sonnet-4.6 + +network: + allowed: + - defaults + - dotnet + +tools: + github: + toolsets: [repos, pull_requests] + +safe-outputs: + create-pull-request: + title-prefix: "[drift] " + labels: [instruction-drift, automation] + draft: true + expires: 14 + protected-files: allowed + create-issue: + title-prefix: "[drift] " + labels: [instruction-drift] + close-older-issues: true + expires: 30 + add-comment: + max: 1 + noop: + max: 1 + +concurrency: + group: "instruction-drift-${{ github.run_id }}" + cancel-in-progress: false + +timeout-minutes: 30 +--- + +# Instruction Drift — Detect & Update + +Check whether gh-aw skill files are stale relative to upstream documentation, and if so, create a PR with updates. + +> **🚨 No test messages.** Never call any safe-output tool with placeholder or test content. Every call posts permanently. + +> **🚨 Security:** This workflow has `protected-files: allowed` because it intentionally updates `.github/skills/` files. Review the generated PR carefully before merging. + +## Step 1: Run Staleness Check + +Run the staleness checker against all sync manifests: + +```bash +pwsh .github/skills/instruction-drift/scripts/Check-Staleness.ps1 \ + -SyncManifest .github/instructions/gh-aw-workflows.sync.yaml +``` + +Capture the exit code and full output: +- **Exit 0 (FRESH)** → Go to Step 5 (noop — nothing to do) +- **Exit 1 (STALE)** → Continue to Step 2 +- **Exit 2 (ERROR)** → Create an issue reporting the error, then stop + +## Step 2: Run Upstream Knowledge Extraction + +If stale, scan the github/gh-aw repo for what specifically changed: + +```bash +pwsh .github/skills/instruction-drift/scripts/Scan-GhAwUpdates.ps1 -MaxCommits 50 +``` + +Parse the JSON output. Focus on: +- `new_features` — categorized changes (safe-output, trigger, compiler, security, engine, breaking) +- `safe_output_samples` — real-world patterns from shared/ configs +- `feature_summary` — grouped counts by type + +## Step 3: Analyze and Update + +Read the current skill files: +- `.github/skills/gh-aw-guide/SKILL.md` +- `.github/skills/gh-aw-guide/references/architecture.md` +- `.github/instructions/gh-aw-workflows.instructions.md` +- `.github/instructions/gh-aw-workflows.sync.yaml` + +For each staleness signal from Step 1, cross-reference with the upstream changes from Step 2 to determine what needs updating. + +### Update Rules + +1. **Respect `divergence_sections`** from the sync manifest — NEVER remove or rewrite these sections: + - "Known Limitation: Stale Blocking Reviews" + - "Security Boundaries" + - "Safe Pattern: Checkout + Restore" + - "Common Patterns" + +2. **Classify changes using P0-P3** before editing: + - **P0 (factually wrong)** — Fix immediately. Example: an issue we reference as "open" is now closed. + - **P1 (security-relevant)** — Fix immediately. Example: new anti-pattern or protection mechanism. + - **P2 (new features)** — Add if straightforward. Example: new safe-output type or frontmatter field. + - **P3 (nice-to-have)** — Skip for now. Example: doc reorganization, new examples. + +3. **Only make P0 and P1 changes automatically.** For P2, add a brief note to the PR description listing what could be added. Skip P3 entirely. + +4. **Update the sync manifest** — After making changes: + - Update `last_reviewed` date to today + - Update any `status:` fields on tracked issues that changed state + - Add new tracked issues if the upstream scan discovered relevant ones + +5. **Match existing style** — Use the same formatting, heading structure, and table layout as the existing files. Read the `style:` field in the sync manifest. + +6. **Run the security scanner** after edits to verify no regressions: + ```bash + pwsh .github/skills/gh-aw-guide/scripts/Check-WorkflowSecurity.ps1 + ``` + +### Making Edits + +Use the `edit` tool for all file changes. Make surgical changes — don't rewrite entire sections unless the content is factually wrong. + +Commit each logical change separately: +```bash +git add +git commit -m "docs: update + +Source: + +Co-authored-by: copilot-agentic-workflow[bot] <224017+copilot-agentic-workflow[bot]@users.noreply.github.com>" +``` + +## Step 4: Create PR + +After all edits are committed, the `create-pull-request` safe output will package the changes into a draft PR. + +The PR description should include: +- Which staleness signals triggered the update +- What upstream changes were detected (with commit SHAs) +- What P0/P1 changes were made +- What P2 features were noted but NOT added (for human review) +- The full staleness report output + +## Step 5: No Changes Needed + +If the staleness check returned FRESH (exit 0), call `noop` with a message: + +``` +noop: "All instruction files are fresh — no drift detected. Last checked: " +``` + +Do NOT create issues, PRs, or comments when nothing needs updating. diff --git a/.github/workflows/review-on-open.agent.lock.yml b/.github/workflows/review-on-open.agent.lock.yml index 8b5c25b1c..2a626e898 100644 --- a/.github/workflows/review-on-open.agent.lock.yml +++ b/.github/workflows/review-on-open.agent.lock.yml @@ -26,7 +26,7 @@ # Imports: # - shared/review-shared.md # -# gh-aw-metadata: {"schema_version":"v2","frontmatter_hash":"377cfad596a061f4e7798a69e2a92312ff6364c10927b473c1f77fa311c5d25e","compiler_version":"v0.62.2","strict":true} +# gh-aw-metadata: {"schema_version":"v2","frontmatter_hash":"5b9cb994d50bb1eca293c3962ee93390f8c236c0faef13d26adb76e528755a8e","compiler_version":"v0.62.2","strict":true} name: "Expert Code Review (auto)" "on": @@ -42,8 +42,8 @@ name: "Expert Code Review (auto)" permissions: {} concurrency: + group: "gh-aw-${{ github.workflow }}-${{ github.event.pull_request.number || github.ref || github.run_id }}" cancel-in-progress: true - group: review-${{ github.event.pull_request.number || github.run_id }} run-name: "Expert Code Review (auto)" @@ -329,16 +329,6 @@ jobs: GH_HOST: github.com - name: Install AWF binary run: bash ${RUNNER_TEMP}/gh-aw/actions/install_awf_binary.sh v0.24.3 - - name: Determine automatic lockdown mode for GitHub MCP Server - id: determine-automatic-lockdown - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 - env: - GH_AW_GITHUB_TOKEN: ${{ secrets.GH_AW_GITHUB_TOKEN }} - GH_AW_GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN }} - with: - script: | - const determineAutomaticLockdown = require('${{ runner.temp }}/gh-aw/actions/determine_automatic_lockdown.cjs'); - await determineAutomaticLockdown(github, context, core); - name: Download container images run: bash ${RUNNER_TEMP}/gh-aw/actions/download_docker_images.sh ghcr.io/github/gh-aw-firewall/agent:0.24.3 ghcr.io/github/gh-aw-firewall/api-proxy:0.24.3 ghcr.io/github/gh-aw-firewall/squid:0.24.3 ghcr.io/github/gh-aw-mcpg:v0.1.19 ghcr.io/github/github-mcp-server:v0.32.0 node:lts-alpine - name: Write Safe Outputs Config @@ -347,14 +337,14 @@ jobs: mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/config.json << 'GH_AW_SAFE_OUTPUTS_CONFIG_EOF' - {"add_comment":{"max":5},"create_pull_request_review_comment":{"max":30},"missing_data":{},"missing_tool":{},"noop":{"max":1},"submit_pull_request_review":{"max":1}} + {"add_comment":{"max":5,"target":"*"},"create_pull_request_review_comment":{"max":30},"missing_data":{},"missing_tool":{},"noop":{"max":1},"submit_pull_request_review":{"max":1}} GH_AW_SAFE_OUTPUTS_CONFIG_EOF - name: Write Safe Outputs Tools run: | cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/tools_meta.json << 'GH_AW_SAFE_OUTPUTS_TOOLS_META_EOF' { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 5 comment(s) can be added.", + "add_comment": " CONSTRAINTS: Maximum 5 comment(s) can be added. Target: *.", "create_pull_request_review_comment": " CONSTRAINTS: Maximum 30 review comment(s) can be created. Comments will be on the RIGHT side of the diff.", "submit_pull_request_review": " CONSTRAINTS: Maximum 1 review(s) can be submitted." }, @@ -541,8 +531,6 @@ jobs: GH_AW_SAFE_OUTPUTS: ${{ env.GH_AW_SAFE_OUTPUTS }} GH_AW_SAFE_OUTPUTS_API_KEY: ${{ steps.safe-outputs-start.outputs.api_key }} GH_AW_SAFE_OUTPUTS_PORT: ${{ steps.safe-outputs-start.outputs.port }} - GITHUB_MCP_GUARD_MIN_INTEGRITY: ${{ steps.determine-automatic-lockdown.outputs.min_integrity }} - GITHUB_MCP_GUARD_REPOS: ${{ steps.determine-automatic-lockdown.outputs.repos }} GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} run: | set -eo pipefail @@ -577,8 +565,7 @@ jobs: }, "guard-policies": { "allow-only": { - "min-integrity": "$GITHUB_MCP_GUARD_MIN_INTEGRITY", - "repos": "$GITHUB_MCP_GUARD_REPOS" + "min-integrity": "approved" } } }, @@ -587,13 +574,6 @@ jobs: "url": "http://host.docker.internal:$GH_AW_SAFE_OUTPUTS_PORT", "headers": { "Authorization": "\${GH_AW_SAFE_OUTPUTS_API_KEY}" - }, - "guard-policies": { - "write-sink": { - "accept": [ - "*" - ] - } } } }, @@ -968,7 +948,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.vsblob.vsassets.io,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.nuget.org,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,azuresearch-usnc.nuget.org,azuresearch-ussc.nuget.org,builds.dotnet.microsoft.com,ci.dot.net,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dc.services.visualstudio.com,dist.nuget.org,dot.net,dotnet.microsoft.com,dotnetcli.blob.core.windows.net,github.com,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,nuget.org,nuget.pkg.github.com,nugetregistryv2prod.blob.core.windows.net,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,oneocsp.microsoft.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,pkgs.dev.azure.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com,www.microsoft.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"hide_older_comments\":true,\"max\":5},\"create_pull_request_review_comment\":{\"max\":30,\"side\":\"RIGHT\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"submit_pull_request_review\":{\"max\":1}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"hide_older_comments\":true,\"max\":5,\"target\":\"*\"},\"create_pull_request_review_comment\":{\"max\":30,\"side\":\"RIGHT\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"submit_pull_request_review\":{\"max\":1}}" with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} script: | diff --git a/.github/workflows/review-on-open.agent.md b/.github/workflows/review-on-open.agent.md index f8883cdd3..b00ea6d44 100644 --- a/.github/workflows/review-on-open.agent.md +++ b/.github/workflows/review-on-open.agent.md @@ -11,11 +11,6 @@ permissions: contents: read pull-requests: read -# Intentional: shared group with review.agent.md — a /review cancels in-progress auto-review. -concurrency: - group: "review-${{ github.event.pull_request.number || github.run_id }}" - cancel-in-progress: true - engine: id: copilot model: claude-opus-4.6 diff --git a/.github/workflows/review.agent.lock.yml b/.github/workflows/review.agent.lock.yml index 3cbfb16de..172902812 100644 --- a/.github/workflows/review.agent.lock.yml +++ b/.github/workflows/review.agent.lock.yml @@ -26,7 +26,7 @@ # Imports: # - shared/review-shared.md # -# gh-aw-metadata: {"schema_version":"v2","frontmatter_hash":"bf16a1ced2111c40378c0de269592faf115585b582efd08cbea0ea83f0f7ae3e","compiler_version":"v0.62.2","strict":true} +# gh-aw-metadata: {"schema_version":"v2","frontmatter_hash":"a511843958435876ab898333b0eb77650a94e82b9dd96d1216eea715a454f4f1","compiler_version":"v0.62.2","strict":true} name: "Expert Code Review" "on": @@ -48,8 +48,7 @@ name: "Expert Code Review" permissions: {} concurrency: - cancel-in-progress: true - group: review-${{ github.event.issue.number || inputs.pr_number || github.run_id }} + group: "gh-aw-${{ github.workflow }}-${{ github.event.issue.number || github.event.pull_request.number || github.run_id }}" run-name: "Expert Code Review" @@ -373,16 +372,6 @@ jobs: GH_HOST: github.com - name: Install AWF binary run: bash ${RUNNER_TEMP}/gh-aw/actions/install_awf_binary.sh v0.24.3 - - name: Determine automatic lockdown mode for GitHub MCP Server - id: determine-automatic-lockdown - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 - env: - GH_AW_GITHUB_TOKEN: ${{ secrets.GH_AW_GITHUB_TOKEN }} - GH_AW_GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN }} - with: - script: | - const determineAutomaticLockdown = require('${{ runner.temp }}/gh-aw/actions/determine_automatic_lockdown.cjs'); - await determineAutomaticLockdown(github, context, core); - name: Download container images run: bash ${RUNNER_TEMP}/gh-aw/actions/download_docker_images.sh ghcr.io/github/gh-aw-firewall/agent:0.24.3 ghcr.io/github/gh-aw-firewall/api-proxy:0.24.3 ghcr.io/github/gh-aw-firewall/squid:0.24.3 ghcr.io/github/gh-aw-mcpg:v0.1.19 ghcr.io/github/github-mcp-server:v0.32.0 node:lts-alpine - name: Write Safe Outputs Config @@ -391,14 +380,14 @@ jobs: mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/config.json << 'GH_AW_SAFE_OUTPUTS_CONFIG_EOF' - {"add_comment":{"max":5},"create_pull_request_review_comment":{"max":30},"missing_data":{},"missing_tool":{},"noop":{"max":1},"submit_pull_request_review":{"max":1}} + {"add_comment":{"max":5,"target":"*"},"create_pull_request_review_comment":{"max":30},"missing_data":{},"missing_tool":{},"noop":{"max":1},"submit_pull_request_review":{"max":1}} GH_AW_SAFE_OUTPUTS_CONFIG_EOF - name: Write Safe Outputs Tools run: | cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/tools_meta.json << 'GH_AW_SAFE_OUTPUTS_TOOLS_META_EOF' { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 5 comment(s) can be added.", + "add_comment": " CONSTRAINTS: Maximum 5 comment(s) can be added. Target: *.", "create_pull_request_review_comment": " CONSTRAINTS: Maximum 30 review comment(s) can be created. Comments will be on the RIGHT side of the diff.", "submit_pull_request_review": " CONSTRAINTS: Maximum 1 review(s) can be submitted." }, @@ -585,8 +574,6 @@ jobs: GH_AW_SAFE_OUTPUTS: ${{ env.GH_AW_SAFE_OUTPUTS }} GH_AW_SAFE_OUTPUTS_API_KEY: ${{ steps.safe-outputs-start.outputs.api_key }} GH_AW_SAFE_OUTPUTS_PORT: ${{ steps.safe-outputs-start.outputs.port }} - GITHUB_MCP_GUARD_MIN_INTEGRITY: ${{ steps.determine-automatic-lockdown.outputs.min_integrity }} - GITHUB_MCP_GUARD_REPOS: ${{ steps.determine-automatic-lockdown.outputs.repos }} GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} run: | set -eo pipefail @@ -621,8 +608,7 @@ jobs: }, "guard-policies": { "allow-only": { - "min-integrity": "$GITHUB_MCP_GUARD_MIN_INTEGRITY", - "repos": "$GITHUB_MCP_GUARD_REPOS" + "min-integrity": "approved" } } }, @@ -631,13 +617,6 @@ jobs: "url": "http://host.docker.internal:$GH_AW_SAFE_OUTPUTS_PORT", "headers": { "Authorization": "\${GH_AW_SAFE_OUTPUTS_API_KEY}" - }, - "guard-policies": { - "write-sink": { - "accept": [ - "*" - ] - } } } }, @@ -1041,7 +1020,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.vsblob.vsassets.io,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.nuget.org,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,azuresearch-usnc.nuget.org,azuresearch-ussc.nuget.org,builds.dotnet.microsoft.com,ci.dot.net,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dc.services.visualstudio.com,dist.nuget.org,dot.net,dotnet.microsoft.com,dotnetcli.blob.core.windows.net,github.com,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,nuget.org,nuget.pkg.github.com,nugetregistryv2prod.blob.core.windows.net,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,oneocsp.microsoft.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,pkgs.dev.azure.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com,www.microsoft.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"hide_older_comments\":true,\"max\":5},\"create_pull_request_review_comment\":{\"max\":30,\"side\":\"RIGHT\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"submit_pull_request_review\":{\"max\":1}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"hide_older_comments\":true,\"max\":5,\"target\":\"*\"},\"create_pull_request_review_comment\":{\"max\":30,\"side\":\"RIGHT\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"submit_pull_request_review\":{\"max\":1}}" with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} script: | diff --git a/.github/workflows/review.agent.md b/.github/workflows/review.agent.md index b9ade5eca..5d0d2d973 100644 --- a/.github/workflows/review.agent.md +++ b/.github/workflows/review.agent.md @@ -23,12 +23,6 @@ permissions: contents: read pull-requests: read -# Intentional: shared group across review.agent.md and review-on-open.agent.md -# so a manual /review cancels any in-progress auto-review on the same PR. -concurrency: - group: "review-${{ github.event.issue.number || inputs.pr_number || github.run_id }}" - cancel-in-progress: true - engine: id: copilot model: claude-opus-4.6 diff --git a/.github/workflows/shared/review-shared.md b/.github/workflows/shared/review-shared.md index 4ee575fb0..6d4af5c6c 100644 --- a/.github/workflows/shared/review-shared.md +++ b/.github/workflows/shared/review-shared.md @@ -1,9 +1,9 @@ --- # Shared configuration for expert-review workflows. # -# Imported by review.agent.md (slash command) and any future -# review-on-open.agent.md (pull request opened). Keeps permissions, -# tools, and safe-outputs in one place. +# Imported by review.agent.md (slash command) and review-on-open.agent.md +# (pull request opened). Keeps permissions, tools, and safe-outputs in +# one place so all review entry points share the same behavior. description: "Shared configuration for expert-review workflows" @@ -14,6 +14,7 @@ permissions: tools: github: toolsets: [pull_requests, repos] + min-integrity: approved safe-outputs: create-pull-request-review-comment: @@ -24,13 +25,14 @@ safe-outputs: add-comment: max: 5 hide-older-comments: true + target: "*" --- # Expert Code Review Review pull request #${{ github.event.pull_request.number || github.event.issue.number || inputs.pr_number }} using the `expert-reviewer` agent defined at `.github/agents/expert-reviewer.agent.md`. -> **🚨 No test messages.** Never call any safe-output tool with placeholder or test content. Every call posts permanently on the PR. This applies to you and all sub-agents. +> **No test messages.** Never call any safe-output tool with placeholder or test content. Every call posts permanently on the PR. This applies to you and all sub-agents. ## Instructions @@ -38,13 +40,12 @@ You are the orchestrator. Your job is to dispatch **3 parallel expert-reviewer s ### Step 1: Gather Context -Fetch the PR diff and save it — you will pass it to each sub-agent: +Fetch the PR data using the GitHub MCP tools (not `gh` CLI — credentials are scrubbed inside the agent container). The `tools.github` configuration provides `pull_requests` and `repos` toolsets: -``` -gh pr diff -gh pr view --json title,body -gh pr checks -``` +- Use `get_pull_request` to read the PR title, body, and metadata +- Use `list_pull_request_files` to get the list of changed files +- Use `get_pull_request_diff` to read the full diff +- Use `get_pull_request_reviews` to check existing reviews ### Step 2: Dispatch 3 Parallel Expert Reviewers @@ -84,8 +85,8 @@ Collect findings from all 3 sub-agents and apply consensus: ### Step 4: Validate Paths and Line Numbers Before posting inline comments, validate **both**: -1. **Path**: Run `gh pr diff --name-only` to get the list of files in the diff. Only files in this list can receive inline comments. Comments on other files fail with "Path could not be resolved". -2. **Line**: Parse `@@ -old,len +new,len @@` — the line must be in `[new, new+len)`. Lines outside any hunk fail with "Line could not be resolved". +1. **Path**: Use `list_pull_request_files` MCP tool to get the list of files in the diff. Only files in this list can receive inline comments. Comments on other files fail with "Path could not be resolved". +2. **Line**: Parse `@@ -old,len +new,len @@` from the diff — the line must be in `[new, new+len)`. Lines outside any hunk fail with "Line could not be resolved". **If either path or line is invalid**, move the finding to `add_comment` (design-level) instead. A single invalid inline comment causes the entire `submit_pull_request_review` to fail and ALL inline comments are lost. @@ -98,5 +99,9 @@ Before posting inline comments, validate **both**: - Methodology note: "3 independent reviewers with adversarial consensus" - CI status, test coverage assessment, prior review status - Never mention specific model names — use "Reviewer 1/2/3" - - Always use `event: "COMMENT"` — the review body communicates severity through findings; blocking `REQUEST_CHANGES` reviews can't be auto-dismissed on re-review and cause stale blocks - - **Never use APPROVE or REQUEST_CHANGES** + - `event: "COMMENT"` always — severity is communicated via emoji markers in the body, not the review event type. (Using `REQUEST_CHANGES` causes stale blocking reviews that can't be dismissed — see Known Limitations below.) + - **Never use APPROVE** + +### Known Limitation: Stale Blocking Reviews + +gh-aw does not support `dismiss-pull-request-review` as a safe output, and workflows run with `pull-requests: read` (write is rejected by the compiler). If `REQUEST_CHANGES` were used, a stale blocking review from `github-actions[bot]` would persist even after findings are fixed, requiring manual dismissal. For this reason, all reviews use `COMMENT` event type — severity is expressed via markers in the review body, not the GitHub review state.