Skip to content

perf(security-guard): eliminate redundant API call and raise diff pre-fetch limit to 100 KB#2281

Merged
lpcox merged 3 commits intomainfrom
copilot/analyze-token-usage-report
Apr 29, 2026
Merged

perf(security-guard): eliminate redundant API call and raise diff pre-fetch limit to 100 KB#2281
lpcox merged 3 commits intomainfrom
copilot/analyze-token-usage-report

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 28, 2026

Security Guard averaged ~376K tokens/run because the agent job re-fetched PR file metadata that the gate job already computed, and a 5 KB diff ceiling truncated almost every PR, forcing repeated get_file_contents tool calls.

Changes

  • Remove redundant Check security relevance step — the agent job was duplicating the gh api …/pulls/…/files call already made by check_security_relevance. Replaced with a passthrough from the job output:

    - name: Set security relevance count
      id: security-relevance
      run: |
        echo "security_files_changed=${{ needs.check_security_relevance.outputs.security_files_changed }}" >> "$GITHUB_OUTPUT"
  • Raise DIFF_LIMIT 5 KB → 100 KB — covers the full diff of typical security-relevant PRs, eliminating the get_file_contents round-trips that drove up token counts.

  • Update prompt efficiency guidelines — instruct the agent to batch independent reads in a single tool-use block, work from the pre-fetched diff first, and fall back to mcp__github__get_pull_request_diff (not get_file_contents) only when the diff is truncated.

Lock file regenerated via gh aw compile + post-processing.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/repos/actions/github-script/git/ref/tags/v9
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v0.71.1
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.71.1 --jq [.object.sha, .object.type] | @tsv (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI linked an issue Apr 28, 2026 that may be closed by this pull request
Copilot AI changed the title [WIP] Analyze Claude token usage report for optimization perf(security-guard): eliminate redundant API call and raise diff pre-fetch limit to 100 KB Apr 28, 2026
Copilot AI requested a review from lpcox April 28, 2026 21:02
@lpcox lpcox marked this pull request as ready for review April 28, 2026 23:06
@lpcox lpcox requested a review from Mossaka as a code owner April 28, 2026 23:06
Copilot AI review requested due to automatic review settings April 28, 2026 23:06
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 28, 2026

✅ Coverage Check Passed

Overall Coverage

Metric Base PR Delta
Lines 86.08% 86.16% 📈 +0.08%
Statements 86.08% 86.15% 📈 +0.07%
Functions 88.18% 88.18% ➡️ +0.00%
Branches 80.27% 80.31% 📈 +0.04%
📁 Per-file Coverage Changes (1 files)
File Lines (Before → After) Statements (Before → After)
src/docker-manager.ts 87.4% → 87.7% (+0.29%) 87.0% → 87.3% (+0.27%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR optimizes the Security Guard agentic workflow to reduce token usage and tool round-trips by reusing already-computed PR metadata and increasing the amount of pre-fetched diff content included in the prompt.

Changes:

  • Increase the pre-fetched diff limit from 5 KB → 100 KB and update truncation guidance to use mcp__github__get_pull_request_diff.
  • Remove the redundant “Check security relevance” API call from the agent job and pass through the count from check_security_relevance job outputs.
  • Update prompt efficiency guidance to encourage batching tool reads and using the pre-fetched diff first.
Show a summary per file
File Description
.github/workflows/security-guard.md Raises diff prefetch limit, removes redundant relevance check step, and updates agent prompt guidance.
.github/workflows/security-guard.lock.yml Regenerated compiled workflow reflecting the new diff limit and relevance count passthrough.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (4)

.github/workflows/security-guard.lock.yml:96

  • This uses: reference is no longer pinned to a commit SHA (it uses the mutable tag @v0.71.1). Lock workflows elsewhere in this repo pin github/gh-aw-actions/setup to a full commit SHA; update this to the resolved SHA and recompile the lock file.
        uses: github/gh-aw-actions/setup@v0.71.1
        with:

.github/workflows/security-guard.lock.yml:344

  • This job’s github/gh-aw-actions/setup action is referenced by tag (@v0.71.1) instead of a commit SHA, which defeats the purpose of the compiled lock file. Pin it to the resolved commit SHA and regenerate the lock file so all jobs use the same pinned ref.
        uses: github/gh-aw-actions/setup@v0.71.1
        with:

.github/workflows/security-guard.lock.yml:990

  • This github/gh-aw-actions/setup reference is tag-based (@v0.71.1) rather than SHA-pinned. For lock workflows, please pin to the exact commit SHA (and ensure the manifest/header list are updated accordingly) to avoid supply-chain drift.
        uses: github/gh-aw-actions/setup@v0.71.1
        with:

.github/workflows/security-guard.lock.yml:1112

  • This github/gh-aw-actions/setup uses: entry is tag-based (@v0.71.1) and should be SHA-pinned like other generated *.lock.yml workflows. Re-run compilation in an environment that can resolve tag→SHA (or manually set the resolved SHA) and commit the regenerated lock file.
        uses: github/gh-aw-actions/setup@v0.71.1
        with:
  • Files reviewed: 2/2 changed files
  • Comments generated: 2

Comment on lines +1 to +2
# gh-aw-metadata: {"schema_version":"v3","frontmatter_hash":"5f638c7a1f626a80c808afb016a775ce87a40f8ff2098ab08d778311c7286200","compiler_version":"v0.71.1","strict":true,"agent_id":"claude"}
# gh-aw-manifest: {"version":1,"secrets":["ANTHROPIC_API_KEY","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"373c709c69115d41ff229c7e5df9f8788daa9553","version":"v9"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"v0.71.1","version":"v0.71.1"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.25.29"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.25.29"},{"image":"ghcr.io/github/gh-aw-firewall/cli-proxy:0.25.29"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.25.29"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.0","digest":"sha256:9c2228324fb1f26f39dc9471612e530ae3efc3156dac05efb2e8d212878d454d","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.0@sha256:9c2228324fb1f26f39dc9471612e530ae3efc3156dac05efb2e8d212878d454d"},{"image":"ghcr.io/github/github-mcp-server:v1.0.2","digest":"sha256:26db03408086a99cf1916348dcc4f9614206658f9082a8060dc7c81ad787f4ba","pinned_image":"ghcr.io/github/github-mcp-server:v1.0.2@sha256:26db03408086a99cf1916348dcc4f9614206658f9082a8060dc7c81ad787f4ba"},{"image":"node:lts-alpine","digest":"sha256:d1b3b4da11eefd5941e7f0b9cf17783fc99d9c6fc34884a665f40a06dbdfc94f","pinned_image":"node:lts-alpine@sha256:d1b3b4da11eefd5941e7f0b9cf17783fc99d9c6fc34884a665f40a06dbdfc94f"}]}
Copy link

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gh-aw-manifest now records github/gh-aw-actions/setup with sha:"v0.71.1", which is not a commit SHA. This breaks the lock-file’s action pinning and weakens supply-chain guarantees; resolve the tag to the actual commit SHA (as in other *.lock.yml files) and regenerate the lock file with the pinned SHA values.

This issue also appears in the following locations of the same file:

  • line 95
  • line 343
  • line 989
  • line 1111

Copilot uses AI. Check for mistakes.
Comment thread .github/workflows/security-guard.md Outdated
1. **Review the pre-fetched diff below** (up to 100 KB of changes are included)
2. **Batch all independent reads** in a single tool-use block rather than making sequential calls
3. **Use `mcp__github__get_pull_request_diff`** only when the diff below is truncated and you need the remainder
4. **Use `get_file_contents`** only for files not changed in this PR (e.g., to understand adjacent security context)
Copy link

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The instructions reference get_file_contents, but this workflow’s tool allowlist uses MCP tool names (e.g. mcp__github__get_file_contents). To avoid the agent attempting a non-existent tool, align the doc text/tool name with the actual allowed tool name.

Suggested change
4. **Use `get_file_contents`** only for files not changed in this PR (e.g., to understand adjacent security context)
4. **Use `mcp__github__get_file_contents`** only for files not changed in this PR (e.g., to understand adjacent security context)

Copilot uses AI. Check for mistakes.
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@lpcox
Copy link
Copy Markdown
Collaborator

lpcox commented Apr 28, 2026

@copilot address the review feedback, merge main and recompile the workflows

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 28, 2026

@copilot address the review feedback, merge main and recompile the workflows

Done in da7ca13:

  • Fixed get_file_contentsmcp__github__get_file_contents in the prompt (review comment on line 138)
  • Added github/gh-aw-actions/setup@v0.71.1 → 239aec45b78c8799417efdd5bc6d8cc036629ec1 to .github/aw/actions-lock.json so the compiler resolves the tag to the correct commit SHA; all four uses: lines and the gh-aw-manifest entry in the lock file now carry the pinned SHA
  • Merged main (already up-to-date) and recompiled with post-processing

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/repos/actions/github-script/git/ref/tags/v9
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv s (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v0.71.1
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.71.1 --jq [.object.sha, .object.type] | @tsv (http block)

If you need me to access, download, or install something from one of these locations, you can either:

@github-actions
Copy link
Copy Markdown
Contributor

Smoke Test Results:

Overall: PASS

💥 [THE END] — Illustrated by Smoke Claude

@github-actions
Copy link
Copy Markdown
Contributor

🔬 Smoke Test Results

PR: perf(security-guard): eliminate redundant API call and raise diff pre-fetch limit to 100 KB
Author: @Copilot | Assignees: @lpcox, @Copilot

Test Result
GitHub MCP connectivity
GitHub.com HTTP connectivity ❌ (template vars not expanded)
File write/read ❌ (template vars not expanded)

Overall: FAIL — pre-step template variables ($\{\{ steps.smoke-data.outputs.* }}) were not substituted, so HTTP and file test results are unavailable.

📰 BREAKING: Report filed by Smoke Copilot

@github-actions
Copy link
Copy Markdown
Contributor

Smoke Test: Copilot BYOK (Offline) Mode

Test Result
GitHub MCP connectivity
GitHub.com HTTP ⚠️ template var unresolved
File write/read ❌ file not found
BYOK inference (this response)

Running in BYOK offline mode (COPILOT_OFFLINE=true) via api-proxy → api.githubcopilot.com.

PR author: @Copilot · Assignees: @lpcox, @Copilot

Overall: FAIL — pre-step template variables ($\{\{ steps.smoke-data.outputs.* }}) were not resolved; file test could not be verified.

🔑 BYOK report filed by Smoke Copilot BYOK

@github-actions
Copy link
Copy Markdown
Contributor

fix: block Shocker container-escape syscalls in seccomp profile (CVE-2014-9357)
chore: upgrade to firewall v0.25.29 and mcpg v0.3.1
GitHub MCP review ❌
safeinputs-gh PR query ❌
Playwright GitHub title ✅
Tavily search ❌
File write + cat ✅
Discussion interaction ❌
Build AWF ✅
Overall: FAIL

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex

@github-actions
Copy link
Copy Markdown
Contributor

🏗️ Build Test Suite Results

Ecosystem Project Build/Install Tests Status
Bun elysia 1/1 passed ✅ PASS
Bun hono 1/1 passed ✅ PASS
C++ fmt N/A ✅ PASS
C++ json N/A ✅ PASS
Deno oak N/A 1/1 passed ✅ PASS
Deno std N/A 1/1 passed ✅ PASS
.NET hello-world N/A ✅ PASS
.NET json-parse N/A ✅ PASS
Go color 1/1 passed ✅ PASS
Go env 1/1 passed ✅ PASS
Go uuid 1/1 passed ✅ PASS
Java gson 1/1 passed ✅ PASS
Java caffeine 1/1 passed ✅ PASS
Node.js clsx All passed ✅ PASS
Node.js execa All passed ✅ PASS
Node.js p-limit All passed ✅ PASS
Rust fd 1/1 passed ✅ PASS
Rust zoxide 1/1 passed ✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Note: Java required using a custom Maven local repository path (-s settings.xml) because the pre-configured ~/.m2 directory was owned by root and not writable by the runner user.

Generated by Build Test Suite for issue #2281 · ● 413.9K ·

@lpcox lpcox merged commit 3097d0f into main Apr 29, 2026
60 of 65 checks passed
@lpcox lpcox deleted the copilot/analyze-token-usage-report branch April 29, 2026 00:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

📊 Claude Token Usage Report2026-04-28

3 participants