diff --git a/.github/agents/expert-reviewer.agent.md b/.github/agents/expert-reviewer.agent.md index 34fe8a9fd6..607c4474c5 100644 --- a/.github/agents/expert-reviewer.agent.md +++ b/.github/agents/expert-reviewer.agent.md @@ -67,5 +67,5 @@ Run `gh pr diff --name-only` to get the list of valid paths before post - Findings ranked by severity with consensus markers (e.g., "3/3 reviewers") - CI status, test coverage assessment, prior review status - Never mention specific model names — use "Reviewer 1/2/3" - - `event: "REQUEST_CHANGES"` if any CRITICAL/MODERATE; `event: "COMMENT"` otherwise - - **Never use APPROVE** + - Always use `event: "COMMENT"` — blocking `REQUEST_CHANGES` reviews can't be auto-dismissed on re-review and cause stale blocks (see gh-aw limitation) + - **Never use APPROVE or REQUEST_CHANGES** diff --git a/.github/instructions/gh-aw-workflows.instructions.md b/.github/instructions/gh-aw-workflows.instructions.md index e99022d3f1..ecf9307b80 100644 --- a/.github/instructions/gh-aw-workflows.instructions.md +++ b/.github/instructions/gh-aw-workflows.instructions.md @@ -6,426 +6,29 @@ applyTo: # gh-aw (GitHub Agentic Workflows) Guidelines -## 🚨 Before You Build: Prefer Built-in gh-aw Features - -**CRITICAL RULE:** Before implementing any trigger, output, scheduling, or interaction mechanism in a gh-aw workflow, check whether gh-aw has a built-in feature that does it. gh-aw extends GitHub Actions with many convenience features — manually reimplementing them is always worse (more code, more bugs, missing platform integration like emoji reactions, sanitized inputs, and noise reduction). - -### Step 1: Check the anti-patterns table below -### Step 2: If not listed, check the [triggers reference](https://github.github.com/gh-aw/reference/triggers/), [frontmatter reference](https://github.github.com/gh-aw/reference/frontmatter/), and [safe-outputs reference](https://github.github.com/gh-aw/reference/safe-outputs/) -### Step 3: If a built-in exists, use it. If not, proceed with manual implementation. - -### Anti-Patterns: Manual Reimplementations to Avoid - -| If you're about to implement... | Use this built-in instead | Docs | -|---------------------------------|--------------------------|------| -| `issue_comment` + `startsWith(comment.body, '/cmd')` | `slash_command:` trigger | [Command Triggers](https://github.github.com/gh-aw/reference/command-triggers/) | -| Manual emoji reaction on triggering comment | `reaction:` field under `on:` | [Frontmatter](https://github.github.com/gh-aw/reference/frontmatter/) | -| Posting "workflow started/completed" status comments | `status-comment: true` under `on:` | [Frontmatter](https://github.github.com/gh-aw/reference/frontmatter/) | -| Fixed cron schedule (`0 9 * * 1`) for non-critical timing | `schedule: weekly on monday around 9:00` (fuzzy) | [Triggers](https://github.github.com/gh-aw/reference/triggers/) | -| Manual `if:` to skip bot-authored PRs | `skip-bots:` under `on:` | [Triggers](https://github.github.com/gh-aw/reference/triggers/) | -| Manual `if:` to skip by author role | `skip-roles:` under `on:` | [Triggers](https://github.github.com/gh-aw/reference/triggers/) | -| Manual label check + removal for one-shot commands | `label_command:` trigger | [Triggers](https://github.github.com/gh-aw/reference/triggers/) | -| Editing old comments to collapse them | `hide-older-comments: true` on `add-comment:` | [Safe Outputs](https://github.github.com/gh-aw/reference/safe-outputs/) | -| Creating no-op report issues | `noop: report-as-issue: false` | [Safe Outputs / Monitoring](https://github.github.com/gh-aw/patterns/monitoring/) | -| Auto-closing older issues from same workflow | `close-older-issues: true` on `create-issue:` | [Safe Outputs](https://github.github.com/gh-aw/reference/safe-outputs/) | -| Disabling workflow after a date | `stop-after:` under `on:` | [Triggers](https://github.github.com/gh-aw/reference/triggers/) | -| Manual approval gating | `manual-approval:` under `on:` | [Triggers](https://github.github.com/gh-aw/reference/triggers/) | -| Search-based skip logic in `steps:` | `skip-if-match:` / `skip-if-no-match:` under `on:` | [Triggers](https://github.github.com/gh-aw/reference/triggers/) | -| Locking issues to prevent concurrent edits | `lock-for-agent: true` under trigger | [Triggers](https://github.github.com/gh-aw/reference/triggers/) | -| Manually hiding agent comments | `hide-comment:` safe output | [Safe Outputs](https://github.github.com/gh-aw/reference/safe-outputs/) | -| Custom post-processing jobs for agent output | `safe-outputs.jobs:` custom jobs with MCP tool access | [Custom Safe Outputs](https://github.github.com/gh-aw/reference/custom-safe-outputs/) | -| Wrapping GitHub Actions as agent-callable tools | `safe-outputs.actions:` action wrappers | [Custom Safe Outputs](https://github.github.com/gh-aw/reference/custom-safe-outputs/) | -| Triggering CI on agent-created PRs | `github-token-for-extra-empty-commit:` on `create-pull-request` | [Triggering CI](https://github.github.com/gh-aw/reference/triggering-ci/) | - -**Note:** gh-aw is actively developed. If a capability feels like something a framework would provide natively, check the reference docs — it probably exists even if it's not in this table yet. - -## Architecture - -gh-aw workflows are authored as `.md` files with YAML frontmatter, compiled to `.lock.yml` via `gh aw compile`. The lock file is auto-generated — **never edit it manually**. - -### Execution Model - -``` -activation job (renders prompt from base branch .md via runtime-import) - ↓ ↳ saves .github/ and .agents/ as artifact for later restore -agent job: - user steps: (pre-agent, OUTSIDE firewall, has GITHUB_TOKEN) - ↓ - platform steps: (configure git → checkout_pr_branch.cjs → restore .github/ from artifact → install CLI) - ↓ - pre-agent-steps: (OPTIONAL, runs after checkout but before agent, OUTSIDE firewall) - ↓ - agent: (INSIDE sandboxed container, NO credentials) - ↓ - post-steps: (OPTIONAL, runs after agent completes, OUTSIDE firewall) -``` - -| Context | Has GITHUB_TOKEN | Has gh CLI | Has git creds | Can execute scripts | -|---------|-----------------|-----------|---------------|-------------------| -| `steps:` (user, pre-activation) | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes — **be careful** | -| Platform steps | ✅ Yes | ✅ Yes | ✅ Yes | Platform-controlled | -| `pre-agent-steps:` | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes — runs after checkout | -| Agent container | ❌ Scrubbed | ❌ Scrubbed | ❌ Scrubbed | ✅ But sandboxed | -| `post-steps:` | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes — runs after agent | - -**⚠️ Agent container credential nuance:** `GITHUB_TOKEN` and `gh` CLI credentials are scrubbed inside the agent container. However, `COPILOT_TOKEN` (used for LLM inference) is present in the environment via `--env-all`. Any subprocess (e.g., `dotnet build`, `npm install`) inherits this variable. The AWF network firewall, `redact_secrets.cjs` (post-agent log scrubbing), and the threat detection agent limit the blast radius. See [Security Boundaries](#security-boundaries) below. - -### Step Ordering - -User `steps:` run in the **pre-activation job** (before the agent job starts). Within the agent job, the ordering is: platform steps → `pre-agent-steps:` → agent → `post-steps:`. - -The platform's `checkout_pr_branch.cjs` runs with `if: (github.event.pull_request) || (github.event.issue.pull_request)` — it is **skipped** for `workflow_dispatch` triggers. - -**`pre-agent-steps:`** run after platform checkout and `.github/` restore but before the agent starts. Use these for data preparation that needs the PR branch checked out (e.g., running analysis scripts on PR code). Declared in frontmatter: - -```yaml -pre-agent-steps: - - name: Analyze PR complexity - run: | - echo "Files changed: $(gh pr diff $PR_NUMBER --name-only | wc -l)" > complexity.txt -``` - -**`post-steps:`** run after the agent completes but before safe-outputs. Use these for cleanup, metrics, or post-processing. - -### Prompt Rendering - -The prompt is built in the **activation job** via `{{#runtime-import .github/workflows/.md}}`. This reads the `.md` file from the **base branch** workspace (before any PR checkout). The rendered prompt is uploaded as an artifact and downloaded by the agent job. - -- The agent prompt is always the base branch version — fork PRs cannot alter it -- The prompt references files on disk (e.g., `SKILL.md`) — those files must exist in the agent's workspace - -### Fork PR Activation Gate - -By default, `gh aw compile` automatically injects a fork guard into the activation job's `if:` condition: `head.repo.id == repository_id`. This blocks fork PRs on `pull_request` events. - -To **allow fork PRs**, add `forks: ["*"]` to the `pull_request` trigger in the `.md` frontmatter. The compiler removes the auto-injected guard from the compiled `if:` conditions. This is safe when the workflow uses the `Checkout-GhAwPr.ps1` pattern (checkout + trusted-infra restore) and the agent is sandboxed. - -## Security Boundaries - -### Key Principles (from [GitHub Security Lab](https://securitylab.github.com/resources/github-actions-preventing-pwn-requests/)) - -1. **Never execute untrusted PR code with elevated credentials.** The classic "pwn-request" attack is `pull_request_target` + checkout PR + run build scripts with `GITHUB_TOKEN`. The attack surface includes build scripts (`make`, `build.ps1`), package manager hooks (`npm postinstall`, MSBuild targets), and test runners. - -2. **Treating PR contents as passive data is safe.** Reading, analyzing, or diffing PR code is fine — the danger is *executing* it. Our gh-aw workflows read code for evaluation; they never build or run it. - -3. **`pull_request_target` grants write permissions and secrets access.** This is by design — the workflow YAML comes from the base branch (trusted). But any step that checks out and runs fork code in this context creates a vulnerability. - -4. **`pull_request` from forks has no secrets access.** GitHub withholds secrets because the workflow YAML comes from the fork (untrusted). This is the safe default for CI builds on fork PRs. - -5. **The `workflow_run` pattern separates privilege from code execution.** Build in an unprivileged `pull_request` job → pass artifacts → process in a privileged `workflow_run` job. This is architecturally what gh-aw does: agent runs read-only, `safe_outputs` job has write permissions. - -### gh-aw Defense Layers - -| Layer | What it does | What it doesn't do | -|-------|-------------|-------------------| -| **AWF network firewall** | Restricts outbound to allowlisted domains | Doesn't prevent reading env vars inside the container | -| **`redact_secrets.cjs`** | Scrubs known secret values from logs/artifacts post-agent | Doesn't catch encoded/obfuscated values | -| **Threat detection agent** | Reviews agent outputs before safe-outputs publishes them | Can miss novel exfiltration techniques | -| **Safe-outputs permission separation** | Write operations happen in separate job, not the agent | Agent can still request writes via safe-output tools | -| **Integrity filtering** | Filters untrusted GitHub content before agent sees it (DIFC proxy) | Requires explicit `min-integrity` configuration | -| **Protected files** | Blocks agent from modifying package manifests, `.github/`, etc. | Only applies to `create-pull-request` and `push-to-pull-request-branch` | -| **`max: N` on safe outputs** | Limits number of operations per type | That output could still contain sensitive data (mitigated by redaction) | -| **XPIA prompt** | Instructs LLM to resist prompt injection from untrusted content | LLM compliance is probabilistic, not guaranteed | -| **`pre_activation` role check** | Gates on write-access collaborators | Does not apply if `roles: all` is set | - -### Integrity Filtering - -Integrity filtering (`tools.github.min-integrity`) controls which GitHub content an agent can access during a workflow run. The MCP gateway filters content by trust level before the agent sees it. - -```yaml -tools: - github: - min-integrity: approved - blocked-users: ["known-spammer"] - trusted-users: ["trusted-contributor"] - approval-labels: ["approved-for-agent"] -``` - -**Integrity hierarchy** (highest to lowest): - -| Level | What qualifies | -|-------|---------------| -| `merged` | Merged PRs, commits reachable from default branch | -| `approved` | `OWNER`, `MEMBER`, `COLLABORATOR`; non-fork PRs on public repos; all items in private repos; users in `trusted-users` | -| `unapproved` | `CONTRIBUTOR`, `FIRST_TIME_CONTRIBUTOR` | -| `none` | All content including `FIRST_TIMER` and no-association users | -| `blocked` | Users in `blocked-users` — always denied, cannot be promoted | - -**Recommendation for our workflows:** Use `min-integrity: approved` for workflows that process PR content from external contributors. This prevents prompt injection via untrusted issue comments or PR descriptions. - -### Protected Files (Auto-Enabled) - -When `create-pull-request` or `push-to-pull-request-branch` is configured, protected files are automatically enforced. The agent cannot modify: -- Package manifests (`package.json`, `*.csproj` dependencies, etc.) -- `.github/` directory contents -- Agent instruction files - -Configure behavior with `protected-files:` on the safe output: -- `blocked` (default) — PR creation fails if protected files are modified -- `fallback-to-issue` — PR branch is pushed but an issue is created instead for review -- `allowed` — Disables protection (use with caution) - -### Rules for gh-aw Workflow Authors - -- ✅ **DO** treat PR contents as passive data (read, analyze, diff) -- ✅ **DO** run data-gathering scripts in `steps:` (pre-agent, trusted context) not inside the agent -- ✅ **DO** use `Checkout-GhAwPr.ps1` for `workflow_dispatch` to restore trusted `.github/` from base -- ❌ **DO NOT** run `dotnet build`, `npm install`, or any build command on untrusted PR code inside the agent — build tool hooks (MSBuild targets, postinstall scripts) can read `COPILOT_TOKEN` from the environment -- ❌ **DO NOT** execute workspace scripts (`.ps1`, `.sh`, `.py`) after checking out a fork PR in `steps:` — those run with `GITHUB_TOKEN` -- ❌ **DO NOT** set `roles: all` on workflows that process PR content — this allows any user to trigger the workflow - -## Fork PR Handling - -### The "pwn-request" Threat Model - -The classic attack requires **checkout + execution** of fork code with elevated credentials. Checkout alone is not dangerous — the vulnerability is executing workspace scripts with `GITHUB_TOKEN`. - -Reference: https://securitylab.github.com/resources/github-actions-preventing-pwn-requests/ - -### Platform `.github/` Restore (gh-aw#23769 — Resolved) - -The platform now **automatically preserves `.github/` and `.agents/` from the base branch**. The activation job saves these directories as an artifact, and after `checkout_pr_branch.cjs` checks out the PR branch, the platform restores them from the artifact. Additionally, `.mcp.json` is deleted from the workspace to prevent injection. This means fork PRs can no longer overwrite agent infrastructure (skills, instructions, copilot-instructions) by including modified copies in their branch. - -### Fork PR Behavior by Trigger - -| Trigger | `checkout_pr_branch.cjs` runs? | Fork handling | -|---------|-------------------------------|---------------| -| `pull_request` (default) | ✅ Yes | Blocked by auto-generated activation gate unless `forks: ["*"]` is set | -| `pull_request` + `forks: ["*"]` | ✅ Yes | ✅ Works — platform restores `.github/` from base branch artifact after checkout | -| `workflow_dispatch` | ❌ Skipped | ✅ Works — user steps handle checkout and restore is final | -| `issue_comment` (same-repo) | ✅ Yes | ✅ Works — files already on PR branch | -| `issue_comment` (fork) | ✅ Yes | ✅ Works — platform restores `.github/` from base branch artifact after checkout | -| `slash_command` | ✅ Yes (compiles to `issue_comment` internally) | Same behavior as `issue_comment` above, but with platform-managed command matching, emoji reactions, and sanitized input. Prefer `slash_command:` over manual `issue_comment` + `startsWith()`. | - -### Safe Pattern: Checkout + Restore - -Use the shared `.github/scripts/Checkout-GhAwPr.ps1` script, which implements checkout + restore in a single reusable step: - -```yaml -steps: - - name: Checkout PR and restore agent infrastructure - env: - GH_TOKEN: ${{ github.token }} - PR_NUMBER: ${{ github.event.pull_request.number || inputs.pr_number }} - run: pwsh .github/scripts/Checkout-GhAwPr.ps1 -``` - -The script: -1. Verifies the PR author has write access and rejects fork PRs -2. Captures the base branch SHA before checkout -3. Checks out the PR branch via `gh pr checkout` -4. Restores `.github/skills/`, `.github/instructions/`, and `.github/copilot-instructions.md` from the base branch SHA (fatal on failure) - -**Behavior by trigger:** -- **`workflow_dispatch`**: Platform checkout is skipped, so the script's restore IS the final workspace state (trusted files from base branch) -- **`slash_command`** (same-repo): Platform's `checkout_pr_branch.cjs` handles checkout. Skill files typically match main unless the PR modified them. -- **`slash_command`** (fork): Platform restores `.github/` from base branch artifact after checkout — fork cannot inject modified skills/instructions - -**Note:** While the platform now handles `.github/` restore automatically for fork PRs, our `Checkout-GhAwPr.ps1` script still provides defense-in-depth for `workflow_dispatch` triggers (where platform checkout is skipped) and adds the write-access check that the platform doesn't enforce. - -### Anti-Patterns - -**Do NOT skip checkout for fork PRs:** - -```bash -# ❌ ANTI-PATTERN: Makes fork PRs unevaluable -if [ "$HEAD_OWNER" != "$BASE_OWNER" ]; then - echo "Skipping checkout for fork PR" - exit 0 # Agent evaluates workflow branch instead of PR -fi -``` - -Skipping checkout means the agent evaluates the wrong files. The correct approach is: always check out the PR, then restore agent infrastructure from the base branch. - -**Do NOT execute workspace code after fork checkout:** - -```yaml -# ❌ DANGEROUS: runs fork code with GITHUB_TOKEN -- name: Checkout PR - run: gh pr checkout "$PR_NUMBER" ... -- name: Run analysis - run: pwsh .github/skills/some-script.ps1 -``` - -If you need to run scripts, either: -1. Run them **before** the checkout (from the base branch) -2. Run them **inside the agent container** (sandboxed, no tokens) - -## Compilation - -```bash -# Compile after every change to the .md source -gh aw compile .github/workflows/.md - -# This updates: -# - .github/workflows/.lock.yml (auto-generated) -# - .github/aw/actions-lock.json -``` - -**Always commit the compiled lock file alongside the source `.md`.** - -## Common Patterns - -### Pre-Agent Data Prep (the `steps:` pattern) - -Use `steps:` for any operation requiring GitHub API access that the agent needs: - -```yaml -steps: - - name: Fetch PR data - env: - GH_TOKEN: ${{ github.token }} - run: | - gh pr view "$PR_NUMBER" --json title,body > pr-metadata.json - gh pr diff "$PR_NUMBER" --name-only > changed-files.txt -``` - -### Safe Outputs (Posting Comments) - -```yaml -safe-outputs: - add-comment: - max: 1 - target: "*" # Required for workflow_dispatch (no triggering PR context) -``` - -### Concurrency - -Include all trigger-specific PR number sources: - -```yaml -concurrency: - group: "my-workflow-${{ github.event.issue.number || github.event.pull_request.number || inputs.pr_number || github.run_id }}" - cancel-in-progress: true -``` - -### Noise Reduction - -Filter `pull_request` triggers to relevant paths and add a gate step: - -```yaml -on: - pull_request: - paths: - - 'src/**/tests/**' - -steps: - - name: Gate — skip if no relevant files - if: github.event_name == 'pull_request' - run: | - FILES=$(gh pr diff "$PR_NUMBER" --name-only | grep -E '\.cs$' || true) - if [ -z "$FILES" ]; then exit 1; fi -``` - -Manual triggers (`workflow_dispatch`, `issue_comment`) should bypass the gate. Note: `exit 1` causes a red ❌ on non-matching PRs — this is intentional (no built-in "skip" mechanism in gh-aw steps). - -## Limitations - -| What | Behavior | Workaround | -|------|----------|------------| -| Agent-created PRs don't trigger CI | GitHub Actions ignores pushes from `GITHUB_TOKEN` | Use `github-token-for-extra-empty-commit:` with a PAT or GitHub App token on `create-pull-request`. See [Triggering CI](https://github.github.com/gh-aw/reference/triggering-ci/) | -| `--allow-all-tools` in lock.yml | Emitted by `gh aw compile` | Cannot override from `.md` source | -| `gh` CLI inside agent | Credentials scrubbed | Use `steps:` for API calls, or MCP tools | -| `issue_comment` trigger | Requires workflow on default branch | Must merge to `main` before `/slash-commands` work | -| Duplicate runs | gh-aw sometimes creates 2 runs per dispatch | Harmless, use concurrency groups | -| `slash_command` + `pull_request` in same workflow | Compiler rejects: `cannot use 'slash_command' with 'pull_request' in the same workflow` | Create two separate workflow files importing the same `shared/*.md`. E.g., `review.agent.md` (slash_command) + `review-on-open.agent.md` (pull_request). Both import `shared/review-shared.md` for shared config and orchestration instructions. This is the pattern used by dotnet/msbuild. | - -### Upstream References (All Resolved) - -These issues are now **all closed** — documented here for historical context: - -| Issue | Status | Resolution | -|-------|--------|------------| -| [gh-aw#18481](https://github.com/github/gh-aw/issues/18481) | ✅ Closed | Fork support tracking — umbrella issue, all sub-items shipped | -| [gh-aw#18518](https://github.com/github/gh-aw/issues/18518) | ✅ Closed | `gh aw init` now warns in forks, lists required secrets | -| [gh-aw#18521](https://github.com/github/gh-aw/issues/18521) | ✅ Closed | Fork support docs created — forks are not supported by default; agents will not run on fork PRs unless `forks:` is configured | -| [gh-aw#23769](https://github.com/github/gh-aw/issues/23769) | ✅ Closed | Platform now auto-restores `.github/` and `.agents/` from base branch after checkout; `.mcp.json` deleted to prevent injection | -| [gh-aw#25439](https://github.com/github/gh-aw/issues/25439) | ✅ Closed | `submit-pull-request-review` safe output previously allowed agents to accidentally approve PRs, bypassing branch protection. Resolution: use `allowed-events: [COMMENT, REQUEST_CHANGES]` to block approvals at infrastructure level | - -## Troubleshooting - -| Symptom | Cause | Fix | -|---------|-------|-----| -| Agent evaluates wrong PR | `workflow_dispatch` checks out workflow branch | Add `gh pr checkout` in `steps:` | -| Agent can't find SKILL.md | Fork PR branch doesn't include `.github/skills/` | Platform now restores `.github/` from base branch; ensure workflow uses current compiler version | -| Fork PR skipped on `pull_request` | `forks: ["*"]` not in workflow frontmatter | Add `forks: ["*"]` under `pull_request:` in the `.md` source and recompile | -| `gh` commands fail in agent | Credentials scrubbed inside container | Move to `steps:` section | -| Lock file out of date | Forgot to recompile | Run `gh aw compile` | -| Agent-created PR has no CI checks | `GITHUB_TOKEN` pushes don't trigger Actions | Add `github-token-for-extra-empty-commit:` with a PAT or GitHub App | -| `/slash-command` doesn't trigger | Workflow not on default branch | Merge to `main` first | -| Agent sees stale issue/PR content | Integrity filtering removed it | Check `min-integrity` level; content from `FIRST_TIMER` is filtered at `approved` | -| Protected file error on PR creation | Agent modified `.github/` or package manifests | Set `protected-files: fallback-to-issue` or `allowed` if intentional | - -## Safe Outputs Quick Reference - -Safe outputs enforce security through separation: agents run read-only and request actions via structured output, while separate permission-controlled jobs execute those requests. - -### Available Safe Output Types - -| Category | Types | -|----------|-------| -| **Issues & Discussions** | `create-issue`, `update-issue`, `close-issue`, `link-sub-issue`, `create-discussion`, `update-discussion`, `close-discussion` | -| **Pull Requests** | `create-pull-request`, `update-pull-request`, `close-pull-request`, `create-pull-request-review-comment`, `reply-to-pull-request-review-comment`, `resolve-pull-request-review-thread`, `push-to-pull-request-branch`, `add-reviewer` | -| **Labels & Assignments** | `add-comment`, `hide-comment`, `add-labels`, `remove-labels`, `assign-milestone`, `assign-to-agent`, `assign-to-user`, `unassign-from-user` | -| **Projects & Releases** | `create-project`, `update-project`, `create-project-status-update`, `update-release`, `upload-asset` | -| **Workflow & Security** | `dispatch-workflow`, `call-workflow`, `dispatch_repository`, `create-code-scanning-alert`, `autofix-code-scanning-alert`, `create-agent-session` | -| **System (auto-enabled)** | `noop`, `missing-tool`, `missing-data` | -| **Custom** | `jobs:` (custom post-processing with MCP tool access), `actions:` (GitHub Action wrappers) | - -### Key Safe Output Features for Our Workflows - -**`create-pull-request` notable options:** -- `draft: true` — Enforced as policy (agent cannot override) -- `expires: 14` — Auto-close after 14 days (same-repo only) -- `excluded-files: ["**/*.lock"]` — Strip files from the patch entirely -- `github-token-for-extra-empty-commit:` — Push empty commit with separate token to trigger CI -- `protected-files: fallback-to-issue` — Create issue instead of failing when protected files modified -- `base-branch: "vnext"` — Target non-default branch -- `auto-close-issue: false` — Don't add `Fixes #N` to PR description - -**`add-comment` notable options:** -- `hide-older-comments: true` — Collapse previous comments from same workflow -- `max: N` — Limit comments per run (default: 1) -- `target: "*"` — Required for `workflow_dispatch` (no triggering PR context) - -## Additional Frontmatter Features - -### Source Tracking and Reuse - -```yaml -source: "githubnext/agentics/workflows/ci-doctor.md@v1.0.0" # Track workflow origin -private: true # Prevent installation via gh aw add -resources: # Companion files fetched with gh aw add - - triage-issue.md - - shared/helper-action.yml -labels: ["automation", "ci"] # For gh aw status --label filtering -``` - -### Runtime Overrides - -Override default runtime versions for tools used in workflows: - -```yaml -runtimes: - dotnet: - version: "9.0" - node: - version: "22" - python: - version: "3.12" -``` - -Supported runtimes: `node`, `python`, `go`, `uv`, `bun`, `deno`, `ruby`, `java`, `dotnet`, `elixir`. - -### APM Dependencies - -Import [APM (Agent Package Manager)](https://microsoft.github.io/apm/) packages for shared skills, prompts, and instructions: - -```yaml -imports: - - uses: shared/apm.md - with: - packages: - - microsoft/apm-sample-package - - github/awesome-copilot/skills/review-and-refactor -``` +> **Full documentation:** Read the `gh-aw-guide` skill (`.github/skills/gh-aw-guide/SKILL.md`) for complete coverage including anti-patterns table, common patterns, security boundaries, and architecture deep-dive. + +## Essential Rules + +1. **Never edit `.lock.yml` files** — they're auto-generated by `gh aw compile`. Always commit the lock file alongside the source `.md`. +2. **Prefer built-in gh-aw features** over manual reimplementations — check the [triggers](https://github.github.com/gh-aw/reference/triggers/), [frontmatter](https://github.github.com/gh-aw/reference/frontmatter/), and [safe-outputs](https://github.github.com/gh-aw/reference/safe-outputs/) references first. +3. **Use `slash_command:` trigger** instead of `issue_comment` + `startsWith(comment.body, '/cmd')`. +4. **Use `steps:` for GitHub API calls** — `gh` CLI credentials are scrubbed inside the agent container. +5. **Never execute untrusted PR code** (`dotnet build`, `npm install`) inside the agent — build hooks can read `COPILOT_TOKEN`. +6. **Use `Checkout-GhAwPr.ps1`** for `workflow_dispatch` triggers to check out the PR branch and restore trusted `.github/` from base. +7. **Include all PR number sources** in concurrency groups: `github.event.issue.number || github.event.pull_request.number || inputs.pr_number || github.run_id`. +8. **Block agent PR approvals** with `allowed-events: [COMMENT]` on `submit-pull-request-review` (see rule 10). +9. **Split `slash_command` + `pull_request`** into separate workflow files importing a shared `shared/*.md` — the compiler rejects them in the same file. +10. **COMMENT-only reviews** — use `allowed-events: [COMMENT]` (not `REQUEST_CHANGES`) to avoid stale blocking reviews that can't be auto-dismissed. See the "Known Limitation: Stale Blocking Reviews" section in the `gh-aw-guide` skill. + +## Quick Anti-Pattern Check + +| If you're about to implement... | Use this built-in instead | +|---------------------------------|--------------------------| +| `issue_comment` + `startsWith(comment.body, '/cmd')` | `slash_command:` trigger | +| Manual emoji reaction on triggering comment | `reaction:` field under `on:` | +| Posting "workflow started/completed" status comments | `status-comment: true` under `on:` | +| Editing old comments to collapse them | `hide-older-comments: true` on `add-comment:` | +| Triggering CI on agent-created PRs | `github-token-for-extra-empty-commit:` on `create-pull-request` | + +See `.github/skills/gh-aw-guide/SKILL.md` for the full 19-row anti-patterns table. diff --git a/.github/instructions/gh-aw-workflows.sync.yaml b/.github/instructions/gh-aw-workflows.sync.yaml new file mode 100644 index 0000000000..de1fb1e9be --- /dev/null +++ b/.github/instructions/gh-aw-workflows.sync.yaml @@ -0,0 +1,52 @@ +# Sync manifest for gh-aw workflow instructions +# Used by the instruction-drift skill to detect when upstream docs change. +# Run: pwsh .github/skills/instruction-drift/scripts/Check-Staleness.ps1 -SyncManifest .github/instructions/gh-aw-workflows.sync.yaml + +sync: + target: "../skills/gh-aw-guide/SKILL.md" + secondary_targets: + - "../skills/gh-aw-guide/references/architecture.md" + + # Upstream documentation pages to monitor for changes + reference_urls: + - https://github.github.com/gh-aw/reference/triggers/ + - https://github.github.com/gh-aw/reference/frontmatter/ + - https://github.github.com/gh-aw/reference/safe-outputs/ + - https://github.github.com/gh-aw/reference/command-triggers/ + - https://github.github.com/gh-aw/reference/custom-safe-outputs/ + - https://github.github.com/gh-aw/reference/triggering-ci/ + - https://github.github.com/gh-aw/patterns/monitoring/ + - https://securitylab.github.com/resources/github-actions-preventing-pwn-requests/ + + # GitHub issues whose resolution may require instruction updates + tracked_issues: + - url: https://github.com/github/gh-aw/issues/18481 + status: closed + note: "Fork support tracking — all sub-items shipped" + - url: https://github.com/github/gh-aw/issues/18518 + status: closed + note: "gh aw init now warns in forks" + - url: https://github.com/github/gh-aw/issues/18521 + status: closed + note: "Fork support docs created" + - url: https://github.com/github/gh-aw/issues/23769 + status: closed + note: "Platform auto-restores .github/ from base branch" + - url: https://github.com/github/gh-aw/issues/25439 + status: closed + note: "submit-pull-request-review allowed-events fix shipped" + # PolyPilot-specific: tracking the stale review gap + # gh-aw#25869 — another team independently adopted COMMENT-only workaround + # No upstream issue for supersede-older-reviews yet; monitor gh-aw releases + + # Sections that intentionally diverge from upstream / other repos + divergence_sections: + - "Known Limitation: Stale Blocking Reviews" + - "Security Boundaries" + - "Safe Pattern: Checkout + Restore" + - "Common Patterns" + + # Monitor for new gh-aw platform releases + releases_source: https://github.com/github/gh-aw/releases.atom + + last_reviewed: "2025-07-14" diff --git a/.github/skills/gh-aw-guide/SKILL.md b/.github/skills/gh-aw-guide/SKILL.md new file mode 100644 index 0000000000..f5656de9a0 --- /dev/null +++ b/.github/skills/gh-aw-guide/SKILL.md @@ -0,0 +1,223 @@ +--- +name: gh-aw-guide +description: >- + Comprehensive guide for building and maintaining GitHub Agentic Workflows + (gh-aw). Covers architecture, security boundaries, fork handling, safe + outputs, anti-patterns, compilation, and troubleshooting. Use when creating + or editing gh-aw workflow .md files, writing safe-outputs configurations, + configuring fork PR handling, setting up integrity filtering, debugging + "why doesn't my workflow trigger", or any task involving + .github/workflows/*.md or .lock.yml files. Also use when asked about gh-aw + features, slash commands, pre-agent-steps, protected files, or agentic + workflow security. +--- + +# gh-aw (GitHub Agentic Workflows) Guide + +This skill provides the complete reference for building, securing, and maintaining GitHub Agentic Workflows in this repository. It covers the gh-aw platform's architecture, security model, and all available features. + +## Quick Start + +gh-aw workflows are authored as `.md` files with YAML frontmatter, compiled to `.lock.yml` via `gh aw compile`. The lock file is auto-generated — **never edit it manually**. + +```bash +# Compile after every change to the .md source +gh aw compile .github/workflows/.md + +# This updates: +# - .github/workflows/.lock.yml (auto-generated) +# - .github/aw/actions-lock.json +``` + +**Always commit the compiled lock file alongside the source `.md`.** + +## 🚨 Before You Build: Prefer Built-in gh-aw Features + +**CRITICAL RULE:** Before implementing any trigger, output, scheduling, or interaction mechanism in a gh-aw workflow, check whether gh-aw has a built-in feature that does it. gh-aw extends GitHub Actions with many convenience features — manually reimplementing them is always worse (more code, more bugs, missing platform integration like emoji reactions, sanitized inputs, and noise reduction). + +### Step 1: Check the anti-patterns table below +### Step 2: If not listed, check the [triggers reference](https://github.github.com/gh-aw/reference/triggers/), [frontmatter reference](https://github.github.com/gh-aw/reference/frontmatter/), and [safe-outputs reference](https://github.github.com/gh-aw/reference/safe-outputs/) +### Step 3: If a built-in exists, use it. If not, proceed with manual implementation. + +### Anti-Patterns: Manual Reimplementations to Avoid + +| If you're about to implement... | Use this built-in instead | Docs | +|---------------------------------|--------------------------|------| +| `issue_comment` + `startsWith(comment.body, '/cmd')` | `slash_command:` trigger | [Command Triggers](https://github.github.com/gh-aw/reference/command-triggers/) | +| Manual emoji reaction on triggering comment | `reaction:` field under `on:` | [Frontmatter](https://github.github.com/gh-aw/reference/frontmatter/) | +| Posting "workflow started/completed" status comments | `status-comment: true` under `on:` | [Frontmatter](https://github.github.com/gh-aw/reference/frontmatter/) | +| Fixed cron schedule (`0 9 * * 1`) for non-critical timing | `schedule: weekly on monday around 9:00` (fuzzy) | [Triggers](https://github.github.com/gh-aw/reference/triggers/) | +| Manual `if:` to skip bot-authored PRs | `skip-bots:` under `on:` | [Triggers](https://github.github.com/gh-aw/reference/triggers/) | +| Manual `if:` to skip by author role | `skip-roles:` under `on:` | [Triggers](https://github.github.com/gh-aw/reference/triggers/) | +| Manual label check + removal for one-shot commands | `label_command:` trigger | [Triggers](https://github.github.com/gh-aw/reference/triggers/) | +| Editing old comments to collapse them | `hide-older-comments: true` on `add-comment:` | [Safe Outputs](https://github.github.com/gh-aw/reference/safe-outputs/) | +| Creating no-op report issues | `noop: report-as-issue: false` | [Safe Outputs / Monitoring](https://github.github.com/gh-aw/patterns/monitoring/) | +| Auto-closing older issues from same workflow | `close-older-issues: true` on `create-issue:` | [Safe Outputs](https://github.github.com/gh-aw/reference/safe-outputs/) | +| Disabling workflow after a date | `stop-after:` under `on:` | [Triggers](https://github.github.com/gh-aw/reference/triggers/) | +| Manual approval gating | `manual-approval:` under `on:` | [Triggers](https://github.github.com/gh-aw/reference/triggers/) | +| Search-based skip logic in `steps:` | `skip-if-match:` / `skip-if-no-match:` under `on:` | [Triggers](https://github.github.com/gh-aw/reference/triggers/) | +| Locking issues to prevent concurrent edits | `lock-for-agent: true` under trigger | [Triggers](https://github.github.com/gh-aw/reference/triggers/) | +| Manually hiding agent comments | `hide-comment:` safe output | [Safe Outputs](https://github.github.com/gh-aw/reference/safe-outputs/) | +| Custom post-processing jobs for agent output | `safe-outputs.jobs:` custom jobs with MCP tool access | [Custom Safe Outputs](https://github.github.com/gh-aw/reference/custom-safe-outputs/) | +| Wrapping GitHub Actions as agent-callable tools | `safe-outputs.actions:` action wrappers | [Custom Safe Outputs](https://github.github.com/gh-aw/reference/custom-safe-outputs/) | +| Triggering CI on agent-created PRs | `github-token-for-extra-empty-commit:` on `create-pull-request` | [Triggering CI](https://github.github.com/gh-aw/reference/triggering-ci/) | +| No guard against agent approving/blocking PRs | `allowed-events: [COMMENT]` on `submit-pull-request-review` — blocks both APPROVE and stale REQUEST_CHANGES | [Safe Outputs](https://github.github.com/gh-aw/reference/safe-outputs/) | + +**Note:** gh-aw is actively developed. If a capability feels like something a framework would provide natively, check the reference docs — it probably exists even if it's not in this table yet. + +For full architecture, security, fork handling, safe outputs, and troubleshooting details, read `references/architecture.md` in this skill directory. + +## Common Patterns + +### Pre-Agent Data Prep (the `steps:` pattern) + +Use `steps:` for any operation requiring GitHub API access that the agent needs: + +```yaml +steps: + - name: Fetch PR data + env: + GH_TOKEN: ${{ github.token }} + run: | + gh pr view "$PR_NUMBER" --json title,body > pr-metadata.json + gh pr diff "$PR_NUMBER" --name-only > changed-files.txt +``` + +### Safe Outputs (Posting Comments) + +```yaml +safe-outputs: + add-comment: + max: 1 + hide-older-comments: true + target: "*" # Required for workflow_dispatch (no triggering PR context) +``` + +### Concurrency + +Include all trigger-specific PR number sources: + +```yaml +concurrency: + group: "my-workflow-${{ github.event.issue.number || github.event.pull_request.number || inputs.pr_number || github.run_id }}" + cancel-in-progress: true +``` + +### Noise Reduction + +Filter `pull_request` triggers to relevant paths and add a gate step: + +```yaml +on: + pull_request: + paths: + - 'src/**/tests/**' + +steps: + - name: Gate — skip if no relevant files + if: github.event_name == 'pull_request' + run: | + FILES=$(gh pr diff "$PR_NUMBER" --name-only | grep -E '\.cs$' || true) + if [ -z "$FILES" ]; then exit 1; fi +``` + +Manual triggers (`workflow_dispatch`, `issue_comment`) should bypass the gate. Note: `exit 1` causes a red ❌ on non-matching PRs — this is intentional (no built-in "skip" mechanism in gh-aw steps). + +### Fork PR Checkout (workflow_dispatch) + +For `workflow_dispatch` workflows that need to evaluate a PR branch: use the shared `Checkout-GhAwPr.ps1` script. It (1) verifies the PR author has write access and rejects fork PRs, (2) checks out the PR branch, and (3) restores `.github/skills/`, `.github/instructions/`, and `.github/copilot-instructions.md` from the base branch SHA — defense-in-depth even though the platform also does this restore automatically. + +```yaml +steps: + - name: Checkout PR and restore agent infrastructure + env: + GH_TOKEN: ${{ github.token }} + PR_NUMBER: ${{ inputs.pr_number }} + run: pwsh .github/scripts/Checkout-GhAwPr.ps1 +``` + +For `pull_request` + fork support (not `workflow_dispatch`): add `forks: ["*"]` to the trigger frontmatter. The platform automatically preserves `.github/` and `.agents/` as a base-branch artifact in the activation job, then restores them after `checkout_pr_branch.cjs` — fork PRs cannot overwrite agent infrastructure (gh-aw#23769, resolved). + +### Security-Critical Patterns + +These four patterns are the most commonly missed when building secure workflows. Use all where applicable: + +**1. Prevent accidental PR approvals** — always restrict review workflows; otherwise the agent can approve PRs and bypass branch protection rules (gh-aw#25439): + +```yaml +safe-outputs: + submit-pull-request-review: + allowed-events: [COMMENT] # Blocks APPROVE and REQUEST_CHANGES — stale blocking reviews can't be auto-dismissed +``` + +**2. CI triggering + protected file safety** for agent-created PRs — `GITHUB_TOKEN` pushes don't trigger CI; a PAT/App token is required. `protected-files` controls what happens when the agent modifies package manifests or `.github/`: + +```yaml +safe-outputs: + create-pull-request: + github-token-for-extra-empty-commit: ${{ secrets.PAT_OR_APP_TOKEN }} # Required to trigger CI + protected-files: fallback-to-issue # Create issue instead of failing if agent touches .github/ or package manifests + # protected-files: blocked (default) | allowed (disables protection) +``` + +**3. Filter untrusted content before the agent sees it** — prevents prompt injection from issue comments or PR descriptions authored by first-timers or contributors: + +```yaml +tools: + github: + min-integrity: approved # Filters FIRST_TIMER / CONTRIBUTOR content; use on workflows that process external PR content +``` + +**4. Fork PR checkout for `workflow_dispatch`** — the platform's `checkout_pr_branch.cjs` is skipped for `workflow_dispatch`, so you **must** use `.github/scripts/Checkout-GhAwPr.ps1` to check out the PR branch, verify write access, reject fork PRs, and restore trusted `.github/` from the base branch. Without it, the agent evaluates the workflow branch instead of the PR: + +```yaml +steps: + - name: Checkout PR and restore agent infrastructure + env: + GH_TOKEN: ${{ github.token }} + PR_NUMBER: ${{ inputs.pr_number }} + run: pwsh .github/scripts/Checkout-GhAwPr.ps1 +``` + +### Frontmatter Features + +```yaml +source: "githubnext/agentics/workflows/ci-doctor.md@v1.0.0" # Track workflow origin +private: true # Prevent installation via gh aw add +resources: # Companion files fetched with gh aw add + - triage-issue.md + - shared/helper-action.yml +labels: ["automation", "ci"] # For gh aw status --label filtering + +runtimes: # Override default runtime versions + dotnet: + version: "9.0" + node: + version: "22" + +imports: # APM package dependencies + - uses: shared/apm.md + with: + packages: + - microsoft/apm-sample-package +``` + +Supported runtimes: `node`, `python`, `go`, `uv`, `bun`, `deno`, `ruby`, `java`, `dotnet`, `elixir`. + +## Known Limitation: Stale Blocking Reviews + +`submit-pull-request-review` with `REQUEST_CHANGES` creates a blocking review that persists even after all findings are fixed and a re-review runs. gh-aw has no `dismiss-pull-request-review` safe output and forbids `pull-requests: write`, so stale bot reviews cannot be auto-dismissed. The `add-comment` output has `hide-older-comments: true` for this lifecycle, but reviews have no equivalent. + +**Our workaround:** Use `allowed-events: [COMMENT]` only — reviews communicate severity via 🔴/🟡/🟢 in the body but never block merging. This loses the GitHub-native "Changes requested" badge and merge-blocking semantics. + +**Upstream request:** A `supersede-older-reviews: true` option on `submit-pull-request-review` would solve this — auto-dismiss previous bot reviews from the same workflow when posting a new one, analogous to `hide-older-comments: true`. See gh-aw#25869 for another team that independently adopted the same COMMENT-only workaround. + +## When to Read the Full Reference + +Read `.github/skills/gh-aw-guide/references/architecture.md` when you need: +- **Execution model** details (step ordering, credential availability, pre-agent-steps/post-steps) +- **Security boundaries** (defense layers, integrity filtering, protected files, rules for authors) +- **Fork PR handling** (platform restore, threat model, trigger-by-trigger behavior) +- **Safe outputs** (complete list of 30+ types, key options for each) +- **Troubleshooting** specific errors +- **Upstream issue history** (all 5 tracked issues and their resolutions) diff --git a/.github/skills/gh-aw-guide/references/architecture.md b/.github/skills/gh-aw-guide/references/architecture.md new file mode 100644 index 0000000000..763103aac8 --- /dev/null +++ b/.github/skills/gh-aw-guide/references/architecture.md @@ -0,0 +1,282 @@ +# gh-aw Architecture Deep-Dive + +This document provides the full architecture, security model, and reference details for GitHub Agentic Workflows (gh-aw). Read the parent `SKILL.md` for the quick-start guide and common patterns. + +## Execution Model + +``` +activation job (renders prompt from base branch .md via runtime-import) + ↓ ↳ saves .github/ and .agents/ as artifact for later restore +agent job: + user steps: (pre-agent, OUTSIDE firewall, has GITHUB_TOKEN) + ↓ + platform steps: (configure git → checkout_pr_branch.cjs → restore .github/ from artifact → install CLI) + ↓ + pre-agent-steps: (OPTIONAL, runs after checkout but before agent, OUTSIDE firewall) + ↓ + agent: (INSIDE sandboxed container, NO credentials) + ↓ + post-steps: (OPTIONAL, runs after agent completes, OUTSIDE firewall) +``` + +| Context | Has GITHUB_TOKEN | Has gh CLI | Has git creds | Can execute scripts | +|---------|-----------------|-----------|---------------|-------------------| +| `steps:` (user, pre-activation) | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes — **be careful** | +| Platform steps | ✅ Yes | ✅ Yes | ✅ Yes | Platform-controlled | +| `pre-agent-steps:` | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes — runs after checkout | +| Agent container | ❌ Scrubbed | ❌ Scrubbed | ❌ Scrubbed | ✅ But sandboxed | +| `post-steps:` | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes — runs after agent | + +**⚠️ Agent container credential nuance:** `GITHUB_TOKEN` and `gh` CLI credentials are scrubbed inside the agent container. However, `COPILOT_TOKEN` (used for LLM inference) is present in the environment via `--env-all`. Any subprocess (e.g., `dotnet build`, `npm install`) inherits this variable. The AWF network firewall, `redact_secrets.cjs` (post-agent log scrubbing), and the threat detection agent limit the blast radius. See [Security Boundaries](#security-boundaries) below. + +## Step Ordering + +User `steps:` run in the **pre-activation job** (before the agent job starts). Within the agent job, the ordering is: platform steps → `pre-agent-steps:` → agent → `post-steps:`. + +The platform's `checkout_pr_branch.cjs` runs with `if: (github.event.pull_request) || (github.event.issue.pull_request)` — it is **skipped** for `workflow_dispatch` triggers. + +**`pre-agent-steps:`** run after platform checkout and `.github/` restore but before the agent starts. Use these for data preparation that needs the PR branch checked out (e.g., running analysis scripts on PR code). Declared in frontmatter: + +```yaml +pre-agent-steps: + - name: Analyze PR complexity + run: | + echo "Files changed: $(gh pr diff $PR_NUMBER --name-only | wc -l)" > complexity.txt +``` + +**`post-steps:`** run after the agent completes but before safe-outputs. Use these for cleanup, metrics, or post-processing. + +## Prompt Rendering + +The prompt is built in the **activation job** via `{{#runtime-import .github/workflows/.md}}`. This reads the `.md` file from the **base branch** workspace (before any PR checkout). The rendered prompt is uploaded as an artifact and downloaded by the agent job. + +- The agent prompt is always the base branch version — fork PRs cannot alter it +- The prompt references files on disk (e.g., `SKILL.md`) — those files must exist in the agent's workspace + +## Fork PR Activation Gate + +By default, `gh aw compile` automatically injects a fork guard into the activation job's `if:` condition: `head.repo.id == repository_id`. This blocks fork PRs on `pull_request` events. + +To **allow fork PRs**, add `forks: ["*"]` to the `pull_request` trigger in the `.md` frontmatter. The compiler removes the auto-injected guard from the compiled `if:` conditions. This is safe when the workflow uses the `Checkout-GhAwPr.ps1` pattern (checkout + trusted-infra restore) and the agent is sandboxed. + +## Security Boundaries + +### Key Principles (from [GitHub Security Lab](https://securitylab.github.com/resources/github-actions-preventing-pwn-requests/)) + +1. **Never execute untrusted PR code with elevated credentials.** The classic "pwn-request" attack is `pull_request_target` + checkout PR + run build scripts with `GITHUB_TOKEN`. The attack surface includes build scripts (`make`, `build.ps1`), package manager hooks (`npm postinstall`, MSBuild targets), and test runners. + +2. **Treating PR contents as passive data is safe.** Reading, analyzing, or diffing PR code is fine — the danger is *executing* it. Our gh-aw workflows read code for evaluation; they never build or run it. + +3. **`pull_request_target` grants write permissions and secrets access.** This is by design — the workflow YAML comes from the base branch (trusted). But any step that checks out and runs fork code in this context creates a vulnerability. + +4. **`pull_request` from forks has no secrets access.** GitHub withholds secrets because the workflow YAML comes from the fork (untrusted). This is the safe default for CI builds on fork PRs. + +5. **The `workflow_run` pattern separates privilege from code execution.** Build in an unprivileged `pull_request` job → pass artifacts → process in a privileged `workflow_run` job. This is architecturally what gh-aw does: agent runs read-only, `safe_outputs` job has write permissions. + +### gh-aw Defense Layers + +| Layer | What it does | What it doesn't do | +|-------|-------------|-------------------| +| **AWF network firewall** | Restricts outbound to allowlisted domains | Doesn't prevent reading env vars inside the container | +| **`redact_secrets.cjs`** | Scrubs known secret values from logs/artifacts post-agent | Doesn't catch encoded/obfuscated values | +| **Threat detection agent** | Reviews agent outputs before safe-outputs publishes them | Can miss novel exfiltration techniques | +| **Safe-outputs permission separation** | Write operations happen in separate job, not the agent | Agent can still request writes via safe-output tools | +| **Integrity filtering** | Filters untrusted GitHub content before agent sees it (DIFC proxy) | Requires explicit `min-integrity` configuration | +| **Protected files** | Blocks agent from modifying package manifests, `.github/`, etc. | Only applies to `create-pull-request` and `push-to-pull-request-branch` | +| **`max: N` on safe outputs** | Limits number of operations per type | That output could still contain sensitive data (mitigated by redaction) | +| **XPIA prompt** | Instructs LLM to resist prompt injection from untrusted content | LLM compliance is probabilistic, not guaranteed | +| **`pre_activation` role check** | Gates on write-access collaborators | Does not apply if `roles: all` is set | + +### Integrity Filtering + +Integrity filtering (`tools.github.min-integrity`) controls which GitHub content an agent can access during a workflow run. The MCP gateway filters content by trust level before the agent sees it. + +```yaml +tools: + github: + min-integrity: approved + blocked-users: ["known-spammer"] + trusted-users: ["trusted-contributor"] + approval-labels: ["approved-for-agent"] +``` + +**Integrity hierarchy** (highest to lowest): + +| Level | What qualifies | +|-------|---------------| +| `merged` | Merged PRs, commits reachable from default branch | +| `approved` | `OWNER`, `MEMBER`, `COLLABORATOR`; non-fork PRs on public repos; all items in private repos; users in `trusted-users` | +| `unapproved` | `CONTRIBUTOR`, `FIRST_TIME_CONTRIBUTOR` | +| `none` | All content including `FIRST_TIMER` and no-association users | +| `blocked` | Users in `blocked-users` — always denied, cannot be promoted | + +**Recommendation for our workflows:** Use `min-integrity: approved` for workflows that process PR content from external contributors. This prevents prompt injection via untrusted issue comments or PR descriptions. + +### Protected Files (Auto-Enabled) + +When `create-pull-request` or `push-to-pull-request-branch` is configured, protected files are automatically enforced. The agent cannot modify: +- Package manifests (`package.json`, `*.csproj` dependencies, etc.) +- `.github/` directory contents +- Agent instruction files + +Configure behavior with `protected-files:` on the safe output: +- `blocked` (default) — PR creation fails if protected files are modified +- `fallback-to-issue` — PR branch is pushed but an issue is created instead for review +- `allowed` — Disables protection (use with caution) + +### Rules for gh-aw Workflow Authors + +- ✅ **DO** treat PR contents as passive data (read, analyze, diff) +- ✅ **DO** run data-gathering scripts in `steps:` (pre-agent, trusted context) not inside the agent +- ✅ **DO** use `Checkout-GhAwPr.ps1` for `workflow_dispatch` to restore trusted `.github/` from base +- ❌ **DO NOT** run `dotnet build`, `npm install`, or any build command on untrusted PR code inside the agent — build tool hooks (MSBuild targets, postinstall scripts) can read `COPILOT_TOKEN` from the environment +- ❌ **DO NOT** execute workspace scripts (`.ps1`, `.sh`, `.py`) after checking out a fork PR in `steps:` — those run with `GITHUB_TOKEN` +- ❌ **DO NOT** set `roles: all` on workflows that process PR content — this allows any user to trigger the workflow + +## Fork PR Handling + +### The "pwn-request" Threat Model + +The classic attack requires **checkout + execution** of fork code with elevated credentials. Checkout alone is not dangerous — the vulnerability is executing workspace scripts with `GITHUB_TOKEN`. + +Reference: https://securitylab.github.com/resources/github-actions-preventing-pwn-requests/ + +### Platform `.github/` Restore (gh-aw#23769 — Resolved) + +The platform now **automatically preserves `.github/` and `.agents/` from the base branch**. The activation job saves these directories as an artifact, and after `checkout_pr_branch.cjs` checks out the PR branch, the platform restores them from the artifact. Additionally, `.mcp.json` is deleted from the workspace to prevent injection. This means fork PRs can no longer overwrite agent infrastructure (skills, instructions, copilot-instructions) by including modified copies in their branch. + +### Fork PR Behavior by Trigger + +| Trigger | `checkout_pr_branch.cjs` runs? | Fork handling | +|---------|-------------------------------|---------------| +| `pull_request` (default) | ✅ Yes | Blocked by auto-generated activation gate unless `forks: ["*"]` is set | +| `pull_request` + `forks: ["*"]` | ✅ Yes | ✅ Works — platform restores `.github/` from base branch artifact after checkout | +| `workflow_dispatch` | ❌ Skipped | ✅ Works — user steps handle checkout and restore is final | +| `issue_comment` (same-repo) | ✅ Yes | ✅ Works — files already on PR branch | +| `issue_comment` (fork) | ✅ Yes | ✅ Works — platform restores `.github/` from base branch artifact after checkout | +| `slash_command` | ✅ Yes (compiles to `issue_comment` internally) | Same behavior as `issue_comment` above, but with platform-managed command matching, emoji reactions, and sanitized input. Prefer `slash_command:` over manual `issue_comment` + `startsWith()`. | + +### Safe Pattern: Checkout + Restore + +Use the shared `.github/scripts/Checkout-GhAwPr.ps1` script, which implements checkout + restore in a single reusable step: + +```yaml +steps: + - name: Checkout PR and restore agent infrastructure + env: + GH_TOKEN: ${{ github.token }} + PR_NUMBER: ${{ github.event.pull_request.number || inputs.pr_number }} + run: pwsh .github/scripts/Checkout-GhAwPr.ps1 +``` + +The script: +1. Verifies the PR author has write access and rejects fork PRs +2. Captures the base branch SHA before checkout +3. Checks out the PR branch via `gh pr checkout` +4. Restores `.github/skills/`, `.github/instructions/`, and `.github/copilot-instructions.md` from the base branch SHA (fatal on failure) + +**Behavior by trigger:** +- **`workflow_dispatch`**: Platform checkout is skipped, so the script's restore IS the final workspace state (trusted files from base branch) +- **`slash_command`** (same-repo): Platform's `checkout_pr_branch.cjs` handles checkout. Skill files typically match main unless the PR modified them. +- **`slash_command`** (fork): Platform restores `.github/` from base branch artifact after checkout — fork cannot inject modified skills/instructions + +**Note:** While the platform now handles `.github/` restore automatically for fork PRs, our `Checkout-GhAwPr.ps1` script still provides defense-in-depth for `workflow_dispatch` triggers (where platform checkout is skipped) and adds the write-access check that the platform doesn't enforce. + +### Anti-Patterns + +**Do NOT skip checkout for fork PRs:** + +```bash +# ❌ ANTI-PATTERN: Makes fork PRs unevaluable +if [ "$HEAD_OWNER" != "$BASE_OWNER" ]; then + echo "Skipping checkout for fork PR" + exit 0 # Agent evaluates workflow branch instead of PR +fi +``` + +Skipping checkout means the agent evaluates the wrong files. The correct approach is: always check out the PR, then restore agent infrastructure from the base branch. + +**Do NOT execute workspace code after fork checkout:** + +```yaml +# ❌ DANGEROUS: runs fork code with GITHUB_TOKEN +- name: Checkout PR + run: gh pr checkout "$PR_NUMBER" ... +- name: Run analysis + run: pwsh .github/skills/some-script.ps1 +``` + +If you need to run scripts, either: +1. Run them **before** the checkout (from the base branch) +2. Run them **inside the agent container** (sandboxed, no tokens) + +## Safe Outputs Quick Reference + +Safe outputs enforce security through separation: agents run read-only and request actions via structured output, while separate permission-controlled jobs execute those requests. + +### Available Safe Output Types + +| Category | Types | +|----------|-------| +| **Issues & Discussions** | `create-issue`, `update-issue`, `close-issue`, `link-sub-issue`, `create-discussion`, `update-discussion`, `close-discussion` | +| **Pull Requests** | `create-pull-request`, `update-pull-request`, `close-pull-request`, `create-pull-request-review-comment`, `reply-to-pull-request-review-comment`, `resolve-pull-request-review-thread`, `push-to-pull-request-branch`, `add-reviewer` | +| **Labels & Assignments** | `add-comment`, `hide-comment`, `add-labels`, `remove-labels`, `assign-milestone`, `assign-to-agent`, `assign-to-user`, `unassign-from-user` | +| **Projects & Releases** | `create-project`, `update-project`, `create-project-status-update`, `update-release`, `upload-asset` | +| **Workflow & Security** | `dispatch-workflow`, `call-workflow`, `dispatch_repository`, `create-code-scanning-alert`, `autofix-code-scanning-alert`, `create-agent-session` | +| **System (auto-enabled)** | `noop`, `missing-tool`, `missing-data` | +| **Custom** | `jobs:` (custom post-processing with MCP tool access), `actions:` (GitHub Action wrappers) | + +### Key Safe Output Features for Our Workflows + +**`create-pull-request` notable options:** +- `draft: true` — Enforced as policy (agent cannot override) +- `expires: 14` — Auto-close after 14 days (same-repo only) +- `excluded-files: ["**/*.lock"]` — Strip files from the patch entirely +- `github-token-for-extra-empty-commit:` — Push empty commit with separate token to trigger CI +- `protected-files: fallback-to-issue` — Create issue instead of failing when protected files modified +- `base-branch: "vnext"` — Target non-default branch +- `auto-close-issue: false` — Don't add `Fixes #N` to PR description + +**`add-comment` notable options:** +- `hide-older-comments: true` — Collapse previous comments from same workflow +- `max: N` — Limit comments per run (default: 1) +- `target: "*"` — Required for `workflow_dispatch` (no triggering PR context) + +## Limitations + +| What | Behavior | Workaround | +|------|----------|------------| +| Agent-created PRs don't trigger CI | GitHub Actions ignores pushes from `GITHUB_TOKEN` | Use `github-token-for-extra-empty-commit:` with a PAT or GitHub App token on `create-pull-request`. See [Triggering CI](https://github.github.com/gh-aw/reference/triggering-ci/) | +| `--allow-all-tools` in lock.yml | Emitted by `gh aw compile` | Cannot override from `.md` source | +| `gh` CLI inside agent | Credentials scrubbed | Use `steps:` for API calls, or MCP tools | +| `issue_comment` trigger | Requires workflow on default branch | Must merge to `main` before `/slash-commands` work | +| Duplicate runs | gh-aw sometimes creates 2 runs per dispatch | Harmless, use concurrency groups | +| `slash_command` + `pull_request` in same workflow | Compiler rejects: `cannot use 'slash_command' with 'pull_request' in the same workflow` | Create two separate workflow files importing the same `shared/*.md`. E.g., `review.agent.md` (slash_command) + `review-on-open.agent.md` (pull_request). Both import `shared/review-shared.md` for shared config and orchestration instructions. | +| Stale `CHANGES_REQUESTED` bot reviews | `REQUEST_CHANGES` creates blocking review that persists after re-review | Use `allowed-events: [COMMENT]` to avoid blocking reviews; manually dismiss stale reviews via API | + +## Upstream References (All Resolved) + +These issues are now **all closed** — documented here for historical context: + +| Issue | Status | Resolution | +|-------|--------|------------| +| [gh-aw#18481](https://github.com/github/gh-aw/issues/18481) | ✅ Closed | Fork support tracking — umbrella issue, all sub-items shipped | +| [gh-aw#18518](https://github.com/github/gh-aw/issues/18518) | ✅ Closed | `gh aw init` now warns in forks, lists required secrets | +| [gh-aw#18521](https://github.com/github/gh-aw/issues/18521) | ✅ Closed | Fork support docs created — forks are not supported by default; agents will not run on fork PRs unless `forks:` is configured | +| [gh-aw#23769](https://github.com/github/gh-aw/issues/23769) | ✅ Closed | Platform now auto-restores `.github/` and `.agents/` from base branch after checkout; `.mcp.json` deleted to prevent injection | +| [gh-aw#25439](https://github.com/github/gh-aw/issues/25439) | ✅ Closed | `submit-pull-request-review` safe output previously allowed agents to accidentally approve PRs, bypassing branch protection. Upstream resolution: `allowed-events` filtering. PolyPilot uses `[COMMENT]` only — see Known Limitation: Stale Blocking Reviews | + +## Troubleshooting + +| Symptom | Cause | Fix | +|---------|-------|-----| +| Agent evaluates wrong PR | `workflow_dispatch` checks out workflow branch | Add `gh pr checkout` in `steps:` | +| Agent can't find SKILL.md | Fork PR branch doesn't include `.github/skills/` | Platform now restores `.github/` from base branch; ensure workflow uses current compiler version | +| Fork PR skipped on `pull_request` | `forks: ["*"]` not in workflow frontmatter | Add `forks: ["*"]` under `pull_request:` in the `.md` source and recompile | +| `gh` commands fail in agent | Credentials scrubbed inside container | Move to `steps:` section | +| Lock file out of date | Forgot to recompile | Run `gh aw compile` | +| Agent-created PR has no CI checks | `GITHUB_TOKEN` pushes don't trigger Actions | Add `github-token-for-extra-empty-commit:` with a PAT or GitHub App | +| `/slash-command` doesn't trigger | Workflow not on default branch | Merge to `main` first | +| Agent sees stale issue/PR content | Integrity filtering removed it | Check `min-integrity` level; content from `FIRST_TIMER` is filtered at `approved` | +| Protected file error on PR creation | Agent modified `.github/` or package manifests | Set `protected-files: fallback-to-issue` or `allowed` if intentional | +| Stale `CHANGES_REQUESTED` review blocks PR | Bot review from earlier run persists after fixes | Use `allowed-events: [COMMENT]` to avoid blocking reviews; manually dismiss stale reviews via API | diff --git a/.github/skills/instruction-drift/SKILL.md b/.github/skills/instruction-drift/SKILL.md new file mode 100644 index 0000000000..ea576212dc --- /dev/null +++ b/.github/skills/instruction-drift/SKILL.md @@ -0,0 +1,74 @@ +--- +name: instruction-drift +description: >- + Detect and report staleness in instruction files that mirror upstream + documentation or cross-repo guidance. Use when checking whether + .github/instructions/*.md or .github/skills/*/SKILL.md are still in sync + with their declared upstream sources. Trigger words: "instruction drift", + "stale instructions", "sync check", "are instructions up to date", + "check freshness", "upstream changes". +--- + +# Instruction-Drift Detection + +This skill provides tooling to detect when instruction files (`.github/instructions/`, `.github/skills/`) have drifted from their declared upstream sources. + +## How It Works + +Each instruction or skill file that mirrors upstream content has a companion `.sync.yaml` file that declares: +- **target**: The local file to check +- **secondary_targets**: Additional files in the same skill directory +- **reference_urls**: Upstream documentation pages to compare against +- **tracked_issues**: GitHub issues whose resolution may require instruction updates +- **divergence_sections**: Sections that intentionally differ from upstream (repo-specific customizations) +- **releases_source**: GitHub releases feed for the upstream project + +## Running a Staleness Check + +```powershell +# Check a specific sync manifest +pwsh .github/skills/instruction-drift/scripts/Check-Staleness.ps1 ` + -SyncManifest .github/instructions/gh-aw-workflows.sync.yaml + +# Check all sync manifests in the repo +Get-ChildItem -Recurse -Filter '*.sync.yaml' .github/ | + ForEach-Object { pwsh .github/skills/instruction-drift/scripts/Check-Staleness.ps1 -SyncManifest $_.FullName } +``` + +## Sync Manifest Schema + +```yaml +# .github/instructions/example.sync.yaml +sync: + target: "../skills/my-skill/SKILL.md" + secondary_targets: + - "../skills/my-skill/references/deep-dive.md" + reference_urls: + - https://docs.example.com/guide + - https://docs.example.com/api + tracked_issues: + - url: https://github.com/org/repo/issues/123 + status: open + note: "Waiting for upstream fix" + divergence_sections: + - "Known Limitation: Custom Section" + - "Repo-Specific Configuration" + releases_source: https://github.com/org/repo/releases.atom + last_reviewed: "2025-07-01" +``` + +## Output + +The script produces a structured report: +- **FRESH** (exit 0): All reference URLs respond 200, no tracked issues changed state vs. manifest, review window not exceeded +- **STALE** (exit 1): One or more signals — reference URLs unreachable, tracked issues changed state vs. manifest expected status, or review window exceeded +- **ERROR** (exit 2): Target file(s) declared in the manifest are missing from disk + +Each signal includes actionable guidance on what to review and update. + +## When to Run + +- Before any PR that modifies gh-aw workflow files +- Periodically (weekly recommended) to catch upstream documentation changes +- After any gh-aw platform release +- When a tracked upstream issue is closed diff --git a/.github/skills/instruction-drift/scripts/Check-Staleness.ps1 b/.github/skills/instruction-drift/scripts/Check-Staleness.ps1 new file mode 100644 index 0000000000..194b8782de --- /dev/null +++ b/.github/skills/instruction-drift/scripts/Check-Staleness.ps1 @@ -0,0 +1,218 @@ +<# +.SYNOPSIS + Checks whether instruction/skill files are stale relative to their + declared upstream sources. + +.DESCRIPTION + Reads a .sync.yaml manifest and checks: + 1. Target file(s) exist on disk + 2. Reference URLs are reachable (HTTP 200) + 3. Tracked issues have not changed status vs. manifest expected state + Reports FRESH, STALE, or ERROR with actionable details. + Note: releases_source is declared in the manifest schema but not yet + checked by this script — planned for a future enhancement. + +.PARAMETER SyncManifest + Path to the .sync.yaml file to check. + +.PARAMETER Verbose + Show detailed output for each check. + +.EXAMPLE + pwsh Check-Staleness.ps1 -SyncManifest .github/instructions/gh-aw-workflows.sync.yaml +#> + +[CmdletBinding()] +param( + [Parameter(Mandatory = $true)] + [string]$SyncManifest +) + +$ErrorActionPreference = 'Stop' + +# --- Helpers --- + +function Read-Yaml { + param([string]$Path) + # Minimal YAML parser for flat sync manifests — handles the subset we use. + # For full YAML, install powershell-yaml module. + $content = Get-Content -Path $Path -Raw + # Return raw content for manual parsing below + return $content +} + +function Test-UrlReachable { + param([string]$Url) + try { + $response = Invoke-WebRequest -Uri $Url -Method Head -TimeoutSec 10 -UseBasicParsing -ErrorAction Stop + return @{ Url = $Url; Status = $response.StatusCode; Ok = ($response.StatusCode -eq 200) } + } + catch { + return @{ Url = $Url; Status = $_.Exception.Message; Ok = $false } + } +} + +function Get-TrackedIssueStatus { + param([string]$Url) + # Extract owner/repo/number from GitHub issue URL + if ($Url -match 'github\.com/([^/]+)/([^/]+)/issues/(\d+)') { + $owner = $Matches[1] + $repo = $Matches[2] + $number = $Matches[3] + try { + $json = gh issue view $number --repo "$owner/$repo" --json state,title --jq '{state: .state, title: .title}' 2>&1 + if ($LASTEXITCODE -eq 0) { + $data = $json | ConvertFrom-Json + return @{ Url = $Url; State = $data.state; Title = $data.title; Ok = $true } + } + } + catch { + Write-Verbose "Failed to check issue $Url`: $_" + } + } + return @{ Url = $Url; State = 'UNKNOWN'; Title = ''; Ok = $false } +} + +# --- Main --- + +if (-not (Test-Path $SyncManifest)) { + Write-Error "Sync manifest not found: $SyncManifest" + exit 1 +} + +$manifestDir = Split-Path -Parent (Resolve-Path $SyncManifest) +$raw = Get-Content -Path $SyncManifest -Raw + +Write-Host "=== Instruction Drift Check ===" -ForegroundColor Cyan +Write-Host "Manifest: $SyncManifest" +Write-Host "" + +$signals = @() +$errors = @() + +# Check target file exists +if ($raw -match 'target:\s*"([^"]+)"') { + $targetPath = Join-Path $manifestDir $Matches[1] + if (Test-Path $targetPath) { + Write-Host "✅ Target exists: $targetPath" -ForegroundColor Green + } + else { + Write-Host "❌ Target MISSING: $targetPath" -ForegroundColor Red + $errors += "Target file missing: $targetPath" + } +} + +# Check secondary targets +$secondaryMatches = [regex]::Matches($raw, 'secondary_targets:\s*\n((?:\s*-\s*"[^"]+"\s*\n?)+)') +if ($secondaryMatches.Count -gt 0) { + $secondaryPaths = [regex]::Matches($secondaryMatches[0].Value, '"([^"]+)"') + foreach ($m in $secondaryPaths) { + $secPath = Join-Path $manifestDir $m.Groups[1].Value + if (Test-Path $secPath) { + Write-Host "✅ Secondary target exists: $secPath" -ForegroundColor Green + } + else { + Write-Host "❌ Secondary target MISSING: $secPath" -ForegroundColor Red + $errors += "Secondary target missing: $secPath" + } + } +} + +# Check reference URLs +$urlMatches = [regex]::Matches($raw, 'reference_urls:\s*\n((?:\s*-\s*https?://[^\s]+\s*\n?)+)') +if ($urlMatches.Count -gt 0) { + $urls = [regex]::Matches($urlMatches[0].Value, '(https?://[^\s]+)') + Write-Host "`nChecking reference URLs..." -ForegroundColor Cyan + foreach ($u in $urls) { + $result = Test-UrlReachable -Url $u.Groups[1].Value + if ($result.Ok) { + Write-Host " ✅ $($result.Url)" -ForegroundColor Green + } + else { + Write-Host " ⚠️ $($result.Url) — $($result.Status)" -ForegroundColor Yellow + $signals += "Reference URL unreachable (may have moved or been removed): $($result.Url)" + } + } +} + +# Check tracked issues — compare actual state vs. manifest expected state +$issueMatches = [regex]::Matches($raw, 'url:\s*(https://github\.com/[^\s]+)\s*\n\s*status:\s*(\w+)') +if ($issueMatches.Count -gt 0) { + Write-Host "`nChecking tracked issues..." -ForegroundColor Cyan + foreach ($im in $issueMatches) { + $issueUrl = $im.Groups[1].Value + $expectedStatus = $im.Groups[2].Value.ToUpper() + $issueResult = Get-TrackedIssueStatus -Url $issueUrl + if ($issueResult.Ok) { + $actualState = $issueResult.State.ToUpper() + $stateIcon = if ($actualState -eq 'CLOSED') { '🔒' } else { '🔓' } + if ($actualState -eq $expectedStatus) { + Write-Host " $stateIcon $($issueResult.Url) — $($issueResult.State) (matches expected)" -ForegroundColor Green + } + elseif ($actualState -eq 'CLOSED' -and $expectedStatus -eq 'OPEN') { + Write-Host " $stateIcon $($issueResult.Url) — CLOSED (was expected OPEN): $($issueResult.Title)" -ForegroundColor Yellow + $signals += "Tracked issue just CLOSED — may need instruction update: $($issueResult.Url)" + } + elseif ($actualState -eq 'OPEN' -and $expectedStatus -eq 'CLOSED') { + Write-Host " $stateIcon $($issueResult.Url) — REOPENED (was expected CLOSED): $($issueResult.Title)" -ForegroundColor Yellow + $signals += "Tracked issue REOPENED — was recorded as closed: $($issueResult.Url)" + } + else { + Write-Host " ⚠️ $($issueResult.Url) — $actualState (expected $expectedStatus)" -ForegroundColor Yellow + $signals += "Tracked issue state mismatch ($actualState vs expected $expectedStatus): $($issueResult.Url)" + } + } + else { + Write-Host " ❓ $issueUrl — could not check" -ForegroundColor Yellow + } + } +} +# Fallback: issues without status field (legacy format) +# Skip URLs already captured by the primary regex above to avoid truncated matches +$primaryUrls = @() +foreach ($pm in $issueMatches) { $primaryUrls += $pm.Groups[1].Value } +$allIssueUrls = [regex]::Matches($raw, '-\s*url:\s*(https://github\.com/[^\s]+)') +if ($allIssueUrls.Count -gt 0) { + foreach ($au in $allIssueUrls) { + $issueUrl = $au.Groups[1].Value + if ($primaryUrls -contains $issueUrl) { continue } + $issueResult = Get-TrackedIssueStatus -Url $issueUrl + if ($issueResult.Ok -and $issueResult.State -eq 'CLOSED') { + Write-Host " 🔒 $($issueResult.Url) — CLOSED (no expected status declared): $($issueResult.Title)" -ForegroundColor Yellow + $signals += "Tracked issue CLOSED (no expected status in manifest): $($issueResult.Url)" + } + } +} + +# Check last_reviewed date +if ($raw -match 'last_reviewed:\s*"(\d{4}-\d{2}-\d{2})"') { + $lastReviewed = [datetime]::Parse($Matches[1]) + $daysSince = ([datetime]::UtcNow - $lastReviewed).Days + Write-Host "`nLast reviewed: $($Matches[1]) ($daysSince days ago)" -ForegroundColor Cyan + if ($daysSince -gt 30) { + $signals += "Last reviewed $daysSince days ago (threshold: 30 days)" + Write-Host " ⚠️ Over 30 days since last review" -ForegroundColor Yellow + } + else { + Write-Host " ✅ Within 30-day review window" -ForegroundColor Green + } +} + +# Summary +Write-Host "`n=== Summary ===" -ForegroundColor Cyan +if ($errors.Count -gt 0) { + Write-Host "Status: ERROR" -ForegroundColor Red + foreach ($e in $errors) { Write-Host " ❌ $e" -ForegroundColor Red } + exit 2 +} +elseif ($signals.Count -gt 0) { + Write-Host "Status: STALE" -ForegroundColor Yellow + foreach ($s in $signals) { Write-Host " ⚠️ $s" -ForegroundColor Yellow } + Write-Host "`nAction: Review the signals above and update instructions if needed." -ForegroundColor Yellow + exit 1 +} +else { + Write-Host "Status: FRESH ✅" -ForegroundColor Green + Write-Host "All checks passed — instructions appear up to date." -ForegroundColor Green + exit 0 +} diff --git a/.github/workflows/review-on-open.agent.lock.yml b/.github/workflows/review-on-open.agent.lock.yml index 30df68ccb4..8b5c25b1cb 100644 --- a/.github/workflows/review-on-open.agent.lock.yml +++ b/.github/workflows/review-on-open.agent.lock.yml @@ -26,7 +26,7 @@ # Imports: # - shared/review-shared.md # -# gh-aw-metadata: {"schema_version":"v2","frontmatter_hash":"e44856fa45e936a539d9ab09e95073b2de1fd509a9e0ed0502087ef3a41323c7","compiler_version":"v0.62.2","strict":true} +# gh-aw-metadata: {"schema_version":"v2","frontmatter_hash":"377cfad596a061f4e7798a69e2a92312ff6364c10927b473c1f77fa311c5d25e","compiler_version":"v0.62.2","strict":true} name: "Expert Code Review (auto)" "on": @@ -42,8 +42,8 @@ name: "Expert Code Review (auto)" permissions: {} concurrency: - group: "gh-aw-${{ github.workflow }}-${{ github.event.pull_request.number || github.ref || github.run_id }}" cancel-in-progress: true + group: review-${{ github.event.pull_request.number || github.run_id }} run-name: "Expert Code Review (auto)" diff --git a/.github/workflows/review-on-open.agent.md b/.github/workflows/review-on-open.agent.md index b00ea6d446..f8883cdd35 100644 --- a/.github/workflows/review-on-open.agent.md +++ b/.github/workflows/review-on-open.agent.md @@ -11,6 +11,11 @@ permissions: contents: read pull-requests: read +# Intentional: shared group with review.agent.md — a /review cancels in-progress auto-review. +concurrency: + group: "review-${{ github.event.pull_request.number || github.run_id }}" + cancel-in-progress: true + engine: id: copilot model: claude-opus-4.6 diff --git a/.github/workflows/review.agent.lock.yml b/.github/workflows/review.agent.lock.yml index 5a1dafe171..3cbfb16de8 100644 --- a/.github/workflows/review.agent.lock.yml +++ b/.github/workflows/review.agent.lock.yml @@ -26,7 +26,7 @@ # Imports: # - shared/review-shared.md # -# gh-aw-metadata: {"schema_version":"v2","frontmatter_hash":"65730ca220c248e6545f004e3b0ee8b5cf91b8cecbdb332bb82033f44ebdd119","compiler_version":"v0.62.2","strict":true} +# gh-aw-metadata: {"schema_version":"v2","frontmatter_hash":"bf16a1ced2111c40378c0de269592faf115585b582efd08cbea0ea83f0f7ae3e","compiler_version":"v0.62.2","strict":true} name: "Expert Code Review" "on": @@ -48,7 +48,8 @@ name: "Expert Code Review" permissions: {} concurrency: - group: "gh-aw-${{ github.workflow }}-${{ github.event.issue.number || github.event.pull_request.number || github.run_id }}" + cancel-in-progress: true + group: review-${{ github.event.issue.number || inputs.pr_number || github.run_id }} run-name: "Expert Code Review" diff --git a/.github/workflows/review.agent.md b/.github/workflows/review.agent.md index 5d0d2d973a..b9ade5eca6 100644 --- a/.github/workflows/review.agent.md +++ b/.github/workflows/review.agent.md @@ -23,6 +23,12 @@ permissions: contents: read pull-requests: read +# Intentional: shared group across review.agent.md and review-on-open.agent.md +# so a manual /review cancels any in-progress auto-review on the same PR. +concurrency: + group: "review-${{ github.event.issue.number || inputs.pr_number || github.run_id }}" + cancel-in-progress: true + engine: id: copilot model: claude-opus-4.6 diff --git a/.github/workflows/shared/review-shared.md b/.github/workflows/shared/review-shared.md index 69cf5faf7d..4ee575fb06 100644 --- a/.github/workflows/shared/review-shared.md +++ b/.github/workflows/shared/review-shared.md @@ -20,7 +20,7 @@ safe-outputs: max: 30 submit-pull-request-review: max: 1 - allowed-events: [COMMENT, REQUEST_CHANGES] + allowed-events: [COMMENT] add-comment: max: 5 hide-older-comments: true @@ -98,5 +98,5 @@ Before posting inline comments, validate **both**: - Methodology note: "3 independent reviewers with adversarial consensus" - CI status, test coverage assessment, prior review status - Never mention specific model names — use "Reviewer 1/2/3" - - `event: "REQUEST_CHANGES"` if any 🔴 CRITICAL or 🟡 MODERATE; `event: "COMMENT"` otherwise - - **Never use APPROVE** + - Always use `event: "COMMENT"` — the review body communicates severity through findings; blocking `REQUEST_CHANGES` reviews can't be auto-dismissed on re-review and cause stale blocks + - **Never use APPROVE or REQUEST_CHANGES**