Skip to content

fix(mcp): audit/audit-diff return graceful JSON errors instead of IsError=true#28291

Merged
pelikhan merged 2 commits intomainfrom
copilot/fix-audit-tool-json-error
Apr 24, 2026
Merged

fix(mcp): audit/audit-diff return graceful JSON errors instead of IsError=true#28291
pelikhan merged 2 commits intomainfrom
copilot/fix-audit-tool-json-error

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 24, 2026

audit and audit-diff MCP tools were setting IsError: true on failure, causing mcp_cli_bridge.cjs to set process.exitCode = 1 and route output to stderr — inconsistent with logs and compile which always return structured JSON with isError=false.

Changes

  • mcp_tools_privileged.go: Set IsError: false in both registerAuditTool and registerAuditDiffTool error handlers; add suggestions field to error envelopes
  • mcp_tools_privileged_test.go: Rename and update SetsIsErrorTrueSetsIsErrorFalse tests; assert suggestions field presence
  • mcp_server_error_codes_test.go: Add explicit result.IsError == false and suggestions assertions
  • mcp_server_json_integration_test.go: Update stale comments and add IsError check

Error envelope shape after this change:

{
  "error": "failed to audit workflow run: ✗ failed to fetch run metadata",
  "run_id_or_url": "9999999999",
  "suggestions": [
    "Verify the run ID is correct",
    "Use the 'logs' tool to list recent run IDs"
  ]
}

audit-diff is fixed with the same pattern for consistency.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/graphql
    • Triggering command: /usr/bin/gh gh repo view --json owner,name --jq .owner.login + "/" + .name 64/pkg/tool/linux_amd64/vet GOINSECURE boring/bbig ache/go/1.25.8/xuser.name 64/pkg/tool/linuTest User (http block)
    • Triggering command: /usr/bin/gh gh repo view owner/repo env 2868015294 7LFx/9x5EhNlMwHDxpQFH7LFx .cfg GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu-test.v=true -c 4998873/b394/_pkg_.a uvTv/8YYGT_gSIAc5Hh4AuvTv .cfg -n1 b/gh-aw/pkg/acti/tmp/test-process-2570702099.js --end-of-optionsNo expressions here ache/go/1.25.8/x64/pkg/tool/linuAdd workflow (http block)
    • Triggering command: /usr/bin/gh gh repo view owner/repo env 2868015294 .cfg .cfg GOINSECURE contextprotocol/init GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu-buildtags -c 01/001/test-frontmatter-with-nes-errorsas log ache/go/1.25.8/x64/pkg/tool/linu-nilfunc -n1 --format=format:config --end-of-options--get-regexp ache/go/1.25.8/x^remote\..*\.gh-resolved$ (http block)
  • https://api.github.com/orgs/test-owner/actions/secrets
    • Triggering command: /usr/bin/gh gh api /orgs/test-owner/actions/secrets --jq .secrets[].name -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /orgs/test-owner/actions/secrets --jq .secrets[].name -c=4 -nolocalimports -importcfg /tmp/go-build2256440870/b392/importcfg -pack /home/REDACTED/work/gh-aw/gh-aw/internal/tools/generate-action-metadata/main.go git rev-�� --show-toplevel git /usr/bin/git --get remote.origin.ur-atomic ache/go/1.25.8/x-bool git (http block)
  • https://api.github.com/repos/actions/ai-inference/git/ref/tags/v1
    • Triggering command: /usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq [.object.sha, .object.type] | @tsv --get-regexp ^remote\..*\.gh-resolved$ /usr/bin/git 6572082/b089/_pkgit .cfg 64/pkg/tool/linu--show-toplevel git rev-�� --show-toplevel 64/pkg/tool/linux_amd64/vet /usr/bin/git 6572082/b040/impnode SK0W/BJGJRDpSI4w/opt/hostedtoolcache/node/24.14.1/x64/bin/npm ache/go/1.25.8/xinstall git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v3
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq [.object.sha, .object.type] | @tsv --show-toplevel x_amd64/compile /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/vet -json GO111MODULE x_amd64/vet /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/vet -ato�� -bool -buildtags /usr/bin/git -errorsas -ifaceassert -nilfunc git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v5
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv 6572082/b202/importcfg -trimpath ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet -p strings -lang=go1.25 ache/go/1.25.8/x--name-only -o st-2550670899/.github/workflows -trimpath 1/x64/bin/node -p compress/gzip -lang=go1.25 ache/go/1.25.8/x64/pkg/tool/linux_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv --show-toplevel -dwarf=false /usr/bin/git go1.25.8 -c=4 -nolocalimports git rev-�� --show-toplevel /tmp/go-build3204998873/b469/_testmain.go /usr/bin/git se 4998873/b245/vetrev-parse ache/go/1.25.8/x--show-toplevel git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv --show-toplevel git /usr/bin/git /tmp/TestParseDegit /tmp/TestParseDerev-parse 1/x64/bin/node git rev-�� --show-toplevel git /usr/bin/git ithub/workflows/git x_amd64/asm /usr/bin/git git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v6
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv /tmp/TestGuardPolicyMinIntegrityOnlymin-integrity_only_defaults_repo2568347081/001 remote /usr/bin/gh hyphen203097501/git hyphen203097501/rev-parse x_amd64/compile gh api /repos/actions/github-script/git/ref/tags/v9 --jq /usr/bin/git -json GO111MODULE x_amd64/vet git (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v8
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq [.object.sha, .object.type] | @tsv --show-toplevel x_amd64/compile /usr/bin/git /tmp/go-build148git -trimpath /opt/hostedtoolc-m git rev-�� --show-toplevel /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linutest@example.com /usr/bin/git /tmp/go-build320git (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v9
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/asm GOINSECURE GOMOD GOMODCACHE x_amd64/asm env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv -json ag.go x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv -json g/catmsg.go x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
  • https://api.github.com/repos/actions/setup-go/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv --git-dir PwQ4sKWnT8ry /usr/bin/gh (http block)
  • https://api.github.com/repos/actions/setup-node/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv /tmp/TestGuardPolicyMinIntegrityOnlymin-integrity_with_explicit_repo627444679/001 config /usr/bin/git remote.origin.urgit GO111MODULE x_amd64/compile git rev-�� mpleWorkflow4024619344/001 x_amd64/compile /usr/bin/git -json GO111MODULE x_amd64/vet git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv --show-toplevel 64/pkg/tool/linux_amd64/vet /opt/hostedtoolcache/node/24.14.1/x64/bin/node ortcfg RR0X2oXnN 64/pkg/tool/linu--show-toplevel /opt/hostedtoolcache/node/24.14.1/x64/bin/node /tmp�� Actor: ${{ github.actor }}, Repo: ${{ github.repository }} 64/pkg/tool/linu-1 /usr/bin/gh ortcfg GO111MODULE 64/pkg/tool/linu--show-toplevel gh (http block)
  • https://api.github.com/repos/actions/upload-artifact/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv ithub-script/git/ref/tags/v9 -extld=gcc ache/go/1.25.8/x64/pkg/tool/linux_amd64/link -json GO111MODULE x_amd64/compile ache/go/1.25.8/x64/pkg/tool/linux_amd64/link conf�� 4998873/b466/typeutil.test remote.origin.url 4998873/b466/importcfg.link -json t/format.go x_amd64/compile 1PwQ4sKWnT8ry/sMr_w3qRcZY0wv0fZFWo/bJFsyISU_LIYuVgol9MA/jtMHmSR1PwQ4sKWnT8ry (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v0.1.2
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.1.2 --jq [.object.sha, .object.type] | @tsv --get remote.origin.url /opt/hostedtoolcache/node/24.14.1/x64/bin/node stall-gh-aw.sh tgit RG6vPflge 64/pkg/tool/linu--show-toplevel /opt/hostedtoolcache/node/24.14.1/x64/bin/node /tmp�� REDACTED.os 64/pkg/tool/linux_amd64/vet /usr/bin/git ortcfg .cfg 64/pkg/tool/linu--show-toplevel git (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.0.0 --jq [.object.sha, .object.type] | @tsv /tmp/gh-aw-test-runs/20260424-145344-33571/test-3317932752/.gith-c=4 rev-parse /usr/bin/git -json GO111MODULE x_amd64/compile git -C /tmp/gh-aw-test-runs/20260424-145344-33571/test-2868015294 remote /usr/bin/git 01 GO111MODULE 64/bin/go git (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv t0 x_amd64/asm (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/1/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/1/artifacts --jq .artifacts[].name vce9/Iw7fHw9tzQV_56Gjvce9 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env 6572082/b232/_pkg_.a taK6/ikh7gQ1RReQdq87ptaK6 eutil.test GOINSECURE contextprotocol/rev-parse GOMODCACHE eutil.test (http block)
    • Triggering command: /usr/bin/gh gh run download 1 --dir test-logs/run-1 .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE fips140/nistec/frev-parse ache/go/1.25.8/x--show-toplevel 64/pkg/tool/linux_amd64/vet env 6572082/b246/_pkg_.a mW0N/BDDpIqgj5QBgNtEbmW0N ache/go/1.25.8/x64/pkg/tool/linu-test.short=true GOINSECURE t/message/catalorev-parse GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linushow (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/1/artifacts --jq .artifacts[].name git /usr/bin/git --show-toplevel ache/go/1.25.8/xrev-parse /usr/bin/git git rev-�� --show-toplevel git /usr/bin/git --show-toplevel ache/go/1.25.8/xrev-parse /usr/bin/git git (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12345/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12345/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env 6572082/b207/_pkg_.a GO111MODULE 64/pkg/tool/linux_amd64/link GOINSECURE g/x/crypto/chachrev-parse GOMODCACHE Vgol9MA/jtMHmSR1PwQ4sKWnT8ry (http block)
    • Triggering command: /usr/bin/gh gh run download 12345 --dir test-logs/run-12345 GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env 4217748878 GO111MODULE .cfg GOINSECURE g/x/text/secure/rev-parse 6572082/b092/sym--show-toplevel ache/go/1.25.8/x64/pkg/tool/linu-buildtags (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12345/artifacts --jq .artifacts[].name Test User /usr/bin/git --show-toplevel ache/go/1.25.8/xremote /usr/bin/git git rev-�� --show-toplevel git /usr/bin/git --show-toplevel ache/go/1.25.8/xrev-parse /usr/bin/git git (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12346/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12346/artifacts --jq .artifacts[].name GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env 2081806404 3cxW/IBlaqeSprCJhOYFQ3cxW 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run download 12346 --dir test-logs/run-12346 GO111MODULE 64/pkg/tool/linu-lang=go1.25 GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linuTest User env 6572082/b223/_pk-c=4 GO111MODULE .cfg GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x/home/REDACTED/work/gh-aw/gh-aw/pkg/typeutil/convert_test.go (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12346/artifacts --jq .artifacts[].name git /usr/bin/git --show-toplevel ache/go/1.25.8/xremote /usr/bin/git git rev-�� --show-toplevel git 64/bin/bash --show-toplevel ache/go/1.25.8/xrev-parse /usr/bin/git git (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/2/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/2/artifacts --jq .artifacts[].name om/modelcontextprotocol/go-sdk@v1.5.0/jsonrpc/js-ifaceassert 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linuTest User env 6572082/b234/_pkg_.a oYmy/n_pwg_VDfKQLamLkoYmy util.test GOINSECURE GOMOD GOMODCACHE util.test (http block)
    • Triggering command: /usr/bin/gh gh run download 2 --dir test-logs/run-2 .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE fips140/edwards2rev-parse ache/go/1.25.8/x--show-toplevel 64/pkg/tool/linux_amd64/vet env 6572082/b248/_pkg_.a .cfg ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE t/internal/formarev-parse GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu--jq (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/2/artifacts --jq .artifacts[].name git /usr/bin/git --show-toplevel x_amd64/compile /usr/bin/git git rev-�� --show-toplevel git eloper-action-main/dist/ripgrep/bin/linux-x64/rg --show-toplevel ache/go/1.25.8/xinit /usr/bin/git eloper-action-main/dist/ripgrep/bin/linux-x64/rg-buildtags (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/3/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/3/artifacts --jq .artifacts[].name bpQE/c2CKE9Q4x2aSUy_HbpQE 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linutest@example.com env 6572082/b242/_pkg_.a V7o_/18xeupG6XnJInX8DV7o_ .cfg GOINSECURE t/internal/tag GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu-trimpath (http block)
    • Triggering command: /usr/bin/gh gh run download 3 --dir test-logs/run-3 .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE sysrand ache/go/1.25.8/x/tmp/go-build3204998873/b474/_pkg_.a 64/pkg/tool/linu-trimpath env 6572082/b236/_pk-p REzZ/UVSmm-gThuygithub.com/github/gh-aw/scripts k GOINSECURE osh-tekuri/jsonsrev-parse GOMODCACHE ache/go/1.25.8/x-goversion (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/3/artifacts --jq .artifacts[].name git /usr/bin/git --show-toplevel ache/go/1.25.8/xrev-parse /usr/bin/git git rev-�� 2864441787/.github/workflows git eloper-action-main/dist/ripgrep/bin/linux-x64/rg --show-toplevel ache/go/1.25.8/xremote /usr/bin/git eloper-action-maorigin (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/4/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/4/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE hlite 6572082/b013/sym--show-toplevel 64/pkg/tool/linux_amd64/vet env 6572082/b241/_pkg_.a _zAe/m6K4S-499xrKjIdi_zAe ache/go/1.25.8/x64/pkg/tool/linu-nolocalimports GOINSECURE t/internal/langurev-parse GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu/tmp/go-build3204998873/b431/_testmain.go (http block)
    • Triggering command: /usr/bin/gh gh run download 4 --dir test-logs/run-4 .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE entropy GOMODCACHE 64/pkg/tool/linux_amd64/vet env 6572082/b235/_pkg_.a h5RJ/fhSiz4P0ozPJ9_2Hh5RJ ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE osh-tekuri/jsons-unsafeptr=false GOMODCACHE ache/go/1.25.8/x/tmp/go-build3204998873/b115/vet.cfg (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/4/artifacts --jq .artifacts[].name git /usr/bin/git --show-toplevel ache/go/1.25.8/xrev-parse /usr/bin/git git rev-�� 2864441787/.github/workflows ROOT" ] && expo /usr/bin/git --show-toplevel ache/go/1.25.8/xconfig /usr/bin/git git (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/5/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/5/artifacts --jq .artifacts[].name rg/x/text@v0.36.-c=4 64/pkg/tool/linu-nolocalimports GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linu/tmp/go-build3204998873/b456/_testmain.go env 6572082/b245/_pkg_.a GO111MODULE .cfg GOINSECURE t/internal/strinrev-parse GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linuremote (http block)
    • Triggering command: /usr/bin/gh gh run download 5 --dir test-logs/run-5 .cfg 64/pkg/tool/linu-buildmode=exe GOINSECURE fips140cache GOMODCACHE 64/pkg/tool/linu-extld=gcc env 6572082/b238/_pkg_.a bmPh/U3cD-KndS88JWpi-bmPh ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE t/feature/pluralrev-parse GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/5/artifacts --jq .artifacts[].name git /usr/bin/git --show-toplevel ache/go/1.25.8/xrev-parse /usr/bin/git git rev-�� 2864441787/.github/workflows git /usr/bin/git --show-toplevel ache/go/1.25.8/xconfig /usr/bin/git r (http block)
  • https://api.github.com/repos/github/gh-aw/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path -c=4 -nolocalimports -importcfg /tmp/go-build3204998873/b423/importcfg -pack /home/REDACTED/work/gh-aw/gh-aw/pkg/gitutil/gitutil.go /home/REDACTED/work/gh-aw/gh-aw/pkg/gitutil/gitutil_test.go env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 100 GOMOD GOMODCACHE x_amd64/vet env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 6 GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env 6572082/b187/_pkg_.a zm1t/ybsydLQ-bM8eUCGDzm1t 64/pkg/tool/linux_amd64/vet GOINSECURE fips140/ed25519 GOMODCACHE 64/pkg/tool/linuremote.origin.url (http block)
  • https://api.github.com/repos/github/gh-aw/contents/.github/workflows/shared/reporting.md
    • Triggering command: /tmp/go-build3204998873/b404/cli.test /tmp/go-build3204998873/b404/cli.test -test.testlogfile=/tmp/go-build3204998873/b404/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true GOINSECURE GOMOD GOMODCACHE x_amd64/asm env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /tmp/go-build2256440870/b404/cli.test /tmp/go-build2256440870/b404/cli.test -test.testlogfile=/tmp/go-build2256440870/b404/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true /tmp/TestHashCongit x_amd64/link /tmp/go-build320--show-toplevel git ache�� --show-toplevel nly (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v0.47.4
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v0.47.4 --jq [.object.sha, .object.type] | @tsv --show-toplevel ache/go/1.25.8/x64/pkg/tool/linuAdd workflow /usr/bin/git e-analyzer.md -trimpath ache/go/1.25.8/x--show-toplevel git rev-�� --show-toplevel ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet /usr/bin/git 5344-33571/test-ls -trimpath 4998873/b129/vet/tmp/gh-aw/aw-feature-branch.patch git (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq [.object.sha, .object.type] | @tsv se 4998873/b011/vet.cfg ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu-importcfg 6572�� DefaultBranchFromLsRemoteWithRea-s DefaultBranchFromLsRemoteWithRea-w .cfg -I /tmp/go-build148-1 -I ache/go/1.25.8/x64/pkg/tool/linuremote2 (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv go GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE SLlQ1ZG/V_z3kiod-goversion (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv --show-toplevel git /usr/bin/git user.email hanges) /usr/bin/git git rev-�� --show-toplevel git /usr/bin/git -3772797823/basegit -3772797823/new.-C /usr/bin/git git (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v2.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/compile GOINSECURE mcpgodebug GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env -json @v1.1.3/base64/base64.go x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv bytealg/indexbyte_wasm.s GO111MODULE x_amd64/vet GOINSECURE GOMOD sm_wasm.s x_amd64/vet 5661�� g_.a GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v3.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env g_.a GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq [.object.sha, .object.type] | @tsv --show-toplevel git /usr/bin/git runs/20260424-14git-receive-pack x_amd64/vet /usr/bin/git git rev-�� --show-toplevel git /usr/bin/git --show-toplevel x_amd64/vet /usr/bin/git git (http block)
  • https://api.github.com/repos/nonexistent/action/git/ref/tags/v999.999.999
    • Triggering command: /usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq [.object.sha, .object.type] | @tsv se 4998873/b006/vet.cfg ger.test GOINSECURE GOMOD GOMODCACHE ger.test 2049�� 6572082/b132/importcfg -trimpath .cfg -p unicode/utf16 -lang=go1.25 ache/go/1.25.8/xv1.0.0 (http block)
  • https://api.github.com/repos/nonexistent/repo/actions/runs/12345
    • Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion GOINSECURE b/gh-aw/pkg/loggrev-parse 6572082/b087/sym--show-toplevel 64/pkg/tool/linu/tmp/go-build3204998873/b462/_testmain.go (http block)
    • Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion --show-toplevel ache/go/1.25.8/xrev-parse /usr/bin/git git rev-�� ithout_min-integrity2667282161/001 git n-dir/bash --show-toplevel ortcfg /usr/bin/git git (http block)
  • https://api.github.com/repos/owner/repo/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh workflow list --repo owner/repo --json name,path,state x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile -o t3040255554/.github/workflows -Eee/499QsILxkBjFfa_H-Eee .cfg -buildmode=exe -buildid=rDZwuyyconfig -s ache/go/1.25.8/x^remote\..*\.gh-resolved$ (http block)
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo /usr/bin/git k/gh-aw/gh-aw/.gbash x_amd64/vet /usr/bin/git git rev-�� --show-toplevel git bin/linux-x64/rg remove upstream /usr/bin/git bin/linux-x64/rg-buildtags (http block)
  • https://api.github.com/repos/test-owner/test-repo/actions/secrets
    • Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name -c=4 (http block)
  • https://api.github.com/repos/test/repo
    • Triggering command: /usr/bin/gh gh api /repos/test/repo --jq .default_branch 6572082/b235/_pkg_.a h5RJ/fhSiz4P0ozPJ9_2Hh5RJ ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE osh-tekuri/jsons-unsafeptr=false GOMODCACHE ache/go/1.25.8/x/tmp/go-build3204998873/b115/vet.cfg (http block)
    • Triggering command: /usr/bin/gh gh api /repos/test/repo --jq .default_branch --show-toplevel git n-dir/bash --show-toplevel ache/go/1.25.8/xrev-parse /usr/bin/git git rev-�� 1490344345/.github/workflows git 64/pkg/tool/linux_amd64/vet --show-toplevel ache/go/1.25.8/xrun /usr/bin/git 64/pkg/tool/linu3 (http block)

If you need me to access, download, or install something from one of these locations, you can either:

…SON errors

- Change IsError from true to false in registerAuditTool error handler
- Change IsError from true to false in registerAuditDiffTool error handler
- Add 'suggestions' field to both error envelopes
- Update unit tests to assert IsError=false and check suggestions field
- Update integration tests to check IsError=false and remove stale comments

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/967df33d-882f-4c79-8d5b-172cddc4013c

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix audit tool to return JSON error for invalid run IDs fix(mcp): audit/audit-diff return graceful JSON errors instead of IsError=true Apr 24, 2026
Copilot AI requested a review from pelikhan April 24, 2026 14:59
@pelikhan pelikhan marked this pull request as ready for review April 24, 2026 15:12
Copilot AI review requested due to automatic review settings April 24, 2026 15:12
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the MCP audit and audit-diff tools to return structured JSON error envelopes with IsError=false, aligning their failure behavior with other MCP tools and avoiding bridge-triggered fatal exits.

Changes:

  • Adjust audit and audit-diff MCP tool error handling to return JSON envelopes with IsError=false.
  • Add a suggestions array to the error envelope payloads for both tools.
  • Update/extend unit and integration tests to assert IsError=false and presence of suggestions.
Show a summary per file
File Description
pkg/cli/mcp_tools_privileged.go Returns graceful JSON error envelopes (with suggestions) and forces IsError=false for audit/audit-diff failures.
pkg/cli/mcp_tools_privileged_test.go Renames/updates tests to assert IsError=false and validate suggestions presence.
pkg/cli/mcp_server_error_codes_test.go Adds assertions that audit returns JSON envelope with IsError=false and includes suggestions.
pkg/cli/mcp_server_json_integration_test.go Updates audit JSON integration expectations and adds an IsError check.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 4/4 changed files
  • Comments generated: 1

Comment on lines 246 to 251
result, err := session.CallTool(ctx, params)
if err != nil {
// Expected behavior: audit command fails with invalid run ID
t.Logf("Audit tool correctly returned error for invalid run ID: %v", err)
// A permission or network error is acceptable in environments without GitHub credentials.
t.Logf("Audit tool returned protocol error (acceptable in CI without credentials): %v", err)
return
}
Copy link

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this integration test, any protocol-level error from session.CallTool is treated as acceptable and the test returns early. That makes the test unable to catch regressions where audit unexpectedly returns an MCP protocol error (the behavior this PR is trying to prevent). Consider only skipping/returning for known CI credential/network conditions (e.g., match specific auth/permission messages like isTestEnvPermissionError does in mcp_server_error_codes_test.go) and failing the test for other errors.

Copilot uses AI. Check for mistakes.
@github-actions github-actions Bot mentioned this pull request Apr 24, 2026
@github-actions
Copy link
Copy Markdown
Contributor

🧪 Test Quality Sentinel Report

Test Quality Score: 100/100

Excellent test quality

Metric Value
New/modified tests analyzed 4
✅ Design tests (behavioral contracts) 4 (100%)
⚠️ Implementation tests (low value) 0 (0%)
Tests with error/edge cases 4 (100%)
Duplicate test clusters 0
Test inflation detected No
🚨 Coding-guideline violations None

Test Classification Details

Test File Classification Issues Detected
TestMCPServer_ErrorCodes_InternalError (modified) pkg/cli/mcp_server_error_codes_test.go ✅ Design None
TestMCPServer_AuditToolReturnsValidJSON (modified) pkg/cli/mcp_server_json_integration_test.go ✅ Design None
TestAuditToolErrorEnvelopeSetsIsErrorFalse (renamed+updated) pkg/cli/mcp_tools_privileged_test.go ✅ Design None
TestAuditDiffToolErrorEnvelopeSetsIsErrorFalse (renamed+updated) pkg/cli/mcp_tools_privileged_test.go ✅ Design None

Language Support

Tests analyzed:

  • 🐹 Go (*_test.go): 4 tests — integration (//go:build integration): 2, unit (//go:build !integration): 2
  • 🟨 JavaScript: no test files changed

Analysis Notes

All four modified tests enforce behavioral contracts of the MCP audit/audit-diff tools:

  • IsError=false assertion: Directly verifies that callers receive a graceful JSON error envelope rather than a fatal MCP protocol error — this is the core behavioral change of the PR and is exactly what a caller would observe.
  • suggestions field assertion: New assertions in three tests verify the response envelope contains actionable guidance, expanding the contract coverage.
  • Build tags: All test files have correct build tags (//go:build integration for integration tests, //go:build !integration for unit tests). ✓
  • Assertion messages: All assertions include descriptive messages. ✓
  • No mock libraries: Tests use real subprocess execution via mockExecCmd (helper process pattern), consistent with the project's "no mock libraries" guideline. ✓

Verdict

Check passed. 0% of new tests are implementation tests (threshold: 30%). The changes sharpen existing behavioral contracts by flipping the IsError expectation and adding coverage for the new suggestions field.


📖 Understanding Test Classifications

Design Tests (High Value) verify what the system does:

  • Assert on observable outputs, return values, or state changes
  • Cover error paths and boundary conditions
  • Would catch a behavioral regression if deleted
  • Remain valid even after internal refactoring

Implementation Tests (Low Value) verify how the system does it:

  • Assert on internal function calls (mocking internals)
  • Only test the happy path with typical inputs
  • Break during legitimate refactoring even when behavior is correct
  • Give false assurance: they pass even when the system is wrong

Goal: Shift toward tests that describe the system's behavioral contract — the promises it makes to its users and collaborators.

References: §24896856466

🧪 Test quality analysis by Test Quality Sentinel · ● 535.1K ·

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Test Quality Sentinel: 100/100. Test quality is excellent — 0% of new tests are implementation tests (threshold: 30%). All 4 modified tests enforce behavioral contracts with proper build tags, descriptive assertion messages, and no mock library violations.

@pelikhan pelikhan merged commit d407af3 into main Apr 24, 2026
40 checks passed
@pelikhan pelikhan deleted the copilot/fix-audit-tool-json-error branch April 24, 2026 15:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[cli-tools-test] audit tool returns MCP isError for invalid run IDs instead of graceful JSON error

3 participants