Skip to content

fix(spec-enforcer): add explicit noop branch when all tests are already up-to-date#28289

Merged
pelikhan merged 2 commits intomainfrom
copilot/debug-package-specification-enforcer
Apr 24, 2026
Merged

fix(spec-enforcer): add explicit noop branch when all tests are already up-to-date#28289
pelikhan merged 2 commits intomainfrom
copilot/debug-package-specification-enforcer

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 24, 2026

The spec-enforcer agent would run for ~9 minutes / 715K tokens and then terminate silently — producing neither a PR nor a noop — when all round-robin-selected packages already had valid spec_test.go files. Phase 5 only described what to do when files changed; there was no "else" branch.

Changes

  • Phase 3 — Renamed "Generate" → "Create or Update"; clarified that existing files should be reviewed against the current README.md and overwritten via edit when stale
  • Phase 5 — Replaced single-branch PR guidance with an explicit two-option decision point:
    • Option A (files changed) → create PR
    • Option B (nothing changed) → call noop with a message listing the reviewed packages
    • Added hard requirement: "Every run MUST end by either creating a PR (Option A) or calling noop (Option B)."
  • Success Criteria — Added noop as an explicitly valid outcome alongside PR creation

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/graphql
    • Triggering command: /usr/bin/gh gh repo view --json owner,name --jq .owner.login + "/" + .name 64/pkg/tool/linux_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh repo view owner/repo k/gh�� 2628279755 on .cfg /../../.prettiergit erignore bin/linux-x64/rg--show-toplevel ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh repo view owner/repo k/gh�� -v on ache/go/1.25.8/x64/pkg/tool/linu-nilfunc /../../.prettiergit erignore (http block)
  • https://api.github.com/orgs/test-owner/actions/secrets
    • Triggering command: /usr/bin/gh gh api /orgs/test-owner/actions/secrets --jq .secrets[].name json' --ignore-p-errorsas (http block)
  • https://api.github.com/repos/actions/ai-inference/git/ref/tags/v1
    • Triggering command: /usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq [.object.sha, .object.type] | @tsv -v 64/pkg/tool/linuremote.origin.url /usr/bin/git y_with_repos_arrgit .cfg 64/pkg/tool/linu--show-toplevel git rev-�� --show-toplevel 64/pkg/tool/linux_amd64/vet /usr/bin/git se 6730335/b017/vet/opt/hostedtoolcache/node/24.14.1/x64/bin/npm ache/go/1.25.8/xinstall git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v3
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq [.object.sha, .object.type] | @tsv -bool -buildtags /usr/bin/git -errorsas -ifaceassert -nilfunc git add /tmp/file-tracker-test3060776055/test1.md /tmp/file-tracker-test3060776055/test2.lock.yml (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v5
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv K7fZbsMw6 --write ache/go/1.25.8/x--limit --ignore-path .prettierignore --log-level=erro--show-toplevel ortcfg -c 2176938454 (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv --show-toplevel -dwarf=false /usr/bin/git go1.25.8 -c=4 -nolocalimports git rev-�� --show-toplevel /tmp/go-build1106730335/b469/_testmain.go /usr/bin/git Onlymin-integritgit 6730335/b267/vetrev-parse ache/go/1.25.8/x--show-toplevel git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv --show-toplevel git-upload-pack /usr/bin/git -stringintconv -tests /usr/bin/git git rev-�� --show-toplevel git /usr/bin/git /tmp/TestGuardPogit rev-parse ache/node/24.14.--show-toplevel git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v6
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv /tmp/TestGuardPolicyMinIntegrityOnlymin-integrity_with_repos=public_3863357982/001 rev-parse /usr/bin/git --get-regexp --global 64/bin/git git -C /home/REDACTED/work/gh-aw/gh-aw/.github/workflows rev-parse /usr/bin/git ../pkg/workflow/git --local x_amd64/vet git (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v8
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq [.object.sha, .object.type] | @tsv --show-toplevel 64/pkg/tool/linux_amd64/compile /usr/bin/git g_.a 6730335/b210/vetcommit ache/go/1.25.8/x-m git rev-�� --show-toplevel ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet /usr/bin/git GWElTwI7T -buildtags /opt/hostedtoolc--show-toplevel git (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v9
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv th .prettierignore --log-level=error --get sh (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv get --local /opt/hostedtoolc-nilfunc credential.helpe/opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/compile (http block)
  • https://api.github.com/repos/actions/setup-go/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv --git-dir 64/pkg/tool/linux_amd64/vet /usr/bin/git le-frontmatter.mgit .cfg 64/pkg/tool/linu--show-toplevel git rev-�� --show-toplevel 64/pkg/tool/linux_amd64/vet /usr/bin/infocmp 3486011122/.githgit (http block)
  • https://api.github.com/repos/actions/setup-node/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv --show-toplevel l /usr/bin/git get --local it git -C mpleWorkflow3714010147/001 l /usr/bin/git ../pkg/workflow/git /deps.go x_amd64/vet git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv /home/REDACTED/work/gh-aw/gh-aw/.github/workflows/api-consumption-report.md 64/pkg/tool/linux_amd64/vet /usr/bin/git ub/workflows .cfg 64/pkg/tool/linu--show-toplevel /usr/bin/git remo�� -v 64/pkg/tool/linux_amd64/vet /usr/bin/infocmp 3506461394/.githgit spec_test.go 64/pkg/tool/linu--show-toplevel infocmp (http block)
  • https://api.github.com/repos/actions/upload-artifact/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv 6730335/b455/_pkg_.a -extld=gcc 6730335/b455=> (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v0.1.2
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.1.2 --jq [.object.sha, .object.type] | @tsv --show-toplevel 64/pkg/tool/linu-test.v=true /usr/bin/gh --noprofile (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.0.0 --jq [.object.sha, .object.type] | @tsv bility_SameInputSameOutput3296844413/001/stability-test.md config /usr/bin/git remote.origin.urgit (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv bility_SameInputSameOutput3296844413/001/stabili-errorsas mkcert /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linu-nilfunc sions.md (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/1/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/1/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run download 1 --dir test-logs/run-1 .cfg 64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12345/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12345/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run download 12345 --dir test-logs/run-12345 .cfg 64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12346/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12346/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run download 12346 --dir test-logs/run-12346 .cfg 64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/2/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/2/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run download 2 --dir test-logs/run-2 .cfg 64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/3/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/3/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run download 3 --dir test-logs/run-3 .cfg 64/pkg/tool/linux_amd64/vet in items: prgit (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/4/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/4/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run download 4 --dir test-logs/run-4 .cfg 64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/5/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/5/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run download 5 --dir test-logs/run-5 .cfg 64/pkg/tool/linu-nolocalimports (http block)
  • https://api.github.com/repos/github/gh-aw/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path -c=4 -nolocalimports -importcfg /tmp/go-build1106730335/b419/importcfg -pack /home/REDACTED/work/gh-aw/gh-aw/pkg/fileutil/fileutil.go /home/REDACTED/work/gh-aw/gh-aw/pkg/fileutil/tar.go conf�� th .prettierigno-errorsas --get p/bin/bash (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 100 (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 6 (http block)
  • https://api.github.com/repos/github/gh-aw/contents/.github/workflows/shared/reporting.md
    • Triggering command: /tmp/go-build1106730335/b404/cli.test /tmp/go-build1106730335/b404/cli.test -test.testlogfile=/tmp/go-build1106730335/b404/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v0.47.4
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v0.47.4 --jq [.object.sha, .object.type] | @tsv --show-toplevel ache/go/1.25.8/x12345 /usr/bin/git 4413/001/stabiligit 6730335/b224/vetrev-parse ache/go/1.25.8/x--show-toplevel git rev-�� --show-toplevel ache/go/1.25.8/xowner/repo /usr/bin/git --write ../../../**/*.js-lh /opt/hostedtoolc/tmp/gh-aw/aw-feature-branch.patch git (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq [.object.sha, .object.type] | @tsv se 6730335/b037/vet.cfg .cfg ./../.prettieriggit -buildid=NFgmU-7init -s ache/go/1.25.8/x64/pkg/tool/linu-importcfg -c "prettier" --write '**/*.cjs' '*-s -extld=gcc .cfg (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv get --local x_amd64/vet gpg.program (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v2.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv get --local x_amd64/vet gpg.program (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv get --local x_amd64/vet commit.gpgsign (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v3.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq [.object.sha, .object.type] | @tsv get --local x_amd64/vet gpg.program (http block)
  • https://api.github.com/repos/nonexistent/action/git/ref/tags/v999.999.999
    • Triggering command: /usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq [.object.sha, .object.type] | @tsv se 6730335/b040/vet.cfg 64/pkg/tool/linux_amd64/vet ./../.prettieriggit (http block)
  • https://api.github.com/repos/nonexistent/repo/actions/runs/12345
    • Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion rkflow/js/**/*.jgit (http block)
  • https://api.github.com/repos/owner/repo/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo tnet/tools/git (http block)
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo /snap/bin/git (http block)
    • Triggering command: /usr/bin/gh gh workflow list --repo owner/repo --json name,path,state 64/pkg/tool/linux_amd64/compile /../../.prettiergit erignore (http block)
  • https://api.github.com/repos/test-owner/test-repo/actions/secrets
    • Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name json' --ignore-path ../../../.pr**/*.json (http block)
  • https://api.github.com/repos/test/repo
    • Triggering command: /usr/bin/gh gh api /repos/test/repo --jq .default_branch 1426794095/custom/workflows (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI linked an issue Apr 24, 2026 that may be closed by this pull request
Copilot AI changed the title [WIP] Fix workflow failure in package specification enforcer fix(spec-enforcer): add explicit noop branch when all tests are already up-to-date Apr 24, 2026
Copilot AI requested a review from pelikhan April 24, 2026 14:45
@pelikhan pelikhan marked this pull request as ready for review April 24, 2026 14:51
Copilot AI review requested due to automatic review settings April 24, 2026 14:51
@pelikhan pelikhan merged commit 3645a84 into main Apr 24, 2026
19 checks passed
@pelikhan pelikhan deleted the copilot/debug-package-specification-enforcer branch April 24, 2026 14:51
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the spec-enforcer workflow instructions to explicitly support a no-op outcome when all selected packages already have up-to-date spec_test.go files, preventing silent termination.

Changes:

  • Renames Phase 3 to “Create or Update” and clarifies how to refresh stale existing spec tests.
  • Adds an explicit Phase 5 decision: create PR when files changed vs call noop when nothing changed.
  • Updates success criteria to treat noop as a valid completion outcome.
Show a summary per file
File Description
.github/workflows/spec-enforcer.md Adds explicit noop branch and clarifies spec test update behavior to ensure every run terminates with a safe output.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 1/1 changed files
  • Comments generated: 2

Comment on lines +343 to +345
### Create Pull Request or Call Noop

If any spec test files were created or updated:
After reviewing all selected packages, choose exactly **one** of the following:
Copy link

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Phase 5 title above this section still says "Save Cache and Create PR", but this section now allows ending the run with noop when nothing changed. Please update the Phase 5 heading to reflect the new two-outcome flow (PR or noop) so the instructions don’t contradict each other.

Copilot uses AI. Check for mistakes.
```

### Create Pull Request
### Create Pull Request or Call Noop
Copy link

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heading uses "Call Noop"; elsewhere the safe-output tool is referred to as noop (lowercase, in backticks), e.g. .github/workflows/shared/noop-reminder.md:1. Consider renaming this heading to use noop for consistency and clarity that it’s a tool name.

Suggested change
### Create Pull Request or Call Noop
### Create Pull Request or Call `noop`

Copilot uses AI. Check for mistakes.
@github-actions github-actions Bot mentioned this pull request Apr 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[aw] Package Specification Enforcer failed

3 participants