perf: optimize secret-digger-claude token usage by lpcox · Pull Request #1894 · github/gh-aw-firewall

lpcox · 2026-04-11T02:19:23Z

Summary

Addresses token optimization recommendations from #1889.

Changes

github: false — disables 52 unused GitHub MCP tools (~26K tokens/run, 67% of cache writes)
max-turns: 8 — hard turn budget (Claude supports this, unlike Copilot)
timeout-minutes 30 → 15 — typical runs complete in ~5 min / 3 turns
Removed duplicate context — Repository, Run ID, Workflow, Engine lines already in system context

Expected Impact

Metric	Before	After	Savings
GitHub MCP tools loaded	52	0	-52 tools
Cache write/run	~39K tokens	~10K tokens	-74%
Cost/run	~$0.51	~$0.18	-64%
Cost/week (16 runs)	~$8.16	~$2.90	-$5.26/wk

Verification

After merging, run via workflow_dispatch and compare token counts in the next token usage report. Target: avg cost/run ≈ $0.18 (±20%).

Closes #1889

Address recommendations from #1889 (Claude token optimization report): 1. Disable GitHub MCP tools (github: false) - 52 unused GitHub MCP tools were loaded (~26K tokens/run) - Workflow only uses bash, cache-memory, and safe-outputs - Estimated savings: ~64% cost reduction ($0.51 → ~$0.18/run) 2. Add max-turns: 8 - Claude engine supports max-turns (unlike Copilot) - Enforces hard turn budget to prevent runaway costs 3. Reduce timeout-minutes from 30 to 15 - Typical runs complete in ~5 minutes with 3 turns - Halves the max cost ceiling for failure scenarios 4. Remove duplicate context from user message - Repository, Run ID, Workflow, Engine lines already injected by gh-aw framework into system context Closes #1889 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions · 2026-04-11T02:20:19Z

✅ Coverage Check Passed

Overall Coverage

Metric	Base	PR	Delta
Lines	85.85%	85.95%	📈 +0.10%
Statements	85.76%	85.85%	📈 +0.09%
Functions	87.54%	87.54%	➡️ +0.00%
Branches	78.56%	78.61%	📈 +0.05%

📁 Per-file Coverage Changes (1 files)

File	Lines (Before → After)	Statements (Before → After)
`src/docker-manager.ts`	86.3% → 86.6% (+0.36%)	85.9% → 86.2% (+0.35%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

Copilot

Pull request overview

Optimizes the secret-digger-claude agent workflow to reduce prompt/tool-schema token overhead and enforce a hard turn/time budget.

Changes:

Disabled GitHub MCP tooling for this workflow (tools.github: false) to avoid loading unused tool schemas.
Added an explicit Claude turn cap (max-turns: 8) and reduced workflow timeout to 15 minutes.
Removed duplicated run-context text and updated the generated lock file to drop GitHub MCP prompt/context and related steps.

Show a summary per file

File	Description
`.github/workflows/secret-digger-claude.md`	Adds `max-turns`, disables GitHub MCP tools, reduces timeout, and trims prompt context.
`.github/workflows/secret-digger-claude.lock.yml`	Regenerates the compiled workflow to remove GitHub MCP server/context and reflect the new budgets.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 2/2 changed files
Comments generated: 2

Copilot · 2026-04-11T02:22:32Z

    BASH_DEFAULT_TIMEOUT_MS: "1800000"  # 30 minutes for bash commands
    BASH_MAX_TIMEOUT_MS: "1800000"      # 30 minutes max timeout


timeout-minutes is now 15, but the engine env still sets BASH_DEFAULT_TIMEOUT_MS / BASH_MAX_TIMEOUT_MS to 30 minutes. This can lead to the agent selecting long-running bash commands that will be terminated by the workflow timeout. Consider lowering the bash timeouts to <= 15 minutes (or increasing timeout-minutes) so tool/runtime limits are aligned.

Suggested change

BASH_DEFAULT_TIMEOUT_MS: "1800000" # 30 minutes for bash commands

BASH_MAX_TIMEOUT_MS: "1800000" # 30 minutes max timeout

BASH_DEFAULT_TIMEOUT_MS: "900000" # 15 minutes for bash commands

BASH_MAX_TIMEOUT_MS: "900000" # 15 minutes max timeout

Copilot · 2026-04-11T02:22:32Z

                "type": "http",
                "url": "http://host.docker.internal:$GH_AW_SAFE_OUTPUTS_PORT",
                "headers": {
                  "Authorization": "$GH_AW_SAFE_OUTPUTS_API_KEY"


The Safe Outputs MCP server config no longer includes a guard-policies.write-sink section. In this repo, other workflows configure Safe Outputs with an explicit write-sink allow list (e.g. .github/workflows/security-guard.lock.yml), and removing it may cause Safe Outputs tool calls (like create_issue) to be blocked or behave differently. Please re-add the Safe Outputs guard policy (or confirm/document why it’s safe to omit here).

Suggested change

"Authorization": "$GH_AW_SAFE_OUTPUTS_API_KEY"

"Authorization": "$GH_AW_SAFE_OUTPUTS_API_KEY"

},

"guard-policies": {

"write-sink": {

"allow": [

"create_issue"

]

}

github-actions · 2026-04-11T02:22:50Z

🔥 Smoke Test Results — PR #1894

Test	Result
GitHub MCP (list merged PRs)	✅ PR #1892 retrieved
GitHub.com connectivity	✅ HTTP 200
File write/read	✅ Pre-step passed

Overall: PASS ✅

Author: @lpcox · No assignees

📰 BREAKING: Report filed by Smoke Copilot

github-actions · 2026-04-11T02:23:02Z

Smoke Test Results

✅ GitHub MCP — "ci: disable threat detection in all workflows" / "perf: disable GitHub MCP tools in secret-digger-claude to cut token costs ~64%"
✅ Playwright — github.com title contains "GitHub"
✅ File Write — /tmp/gh-aw/agent/smoke-test-claude-24272439720.txt created
✅ Bash — file contents verified

Overall: PASS

💥 [THE END] — Illustrated by Smoke Claude

github-actions · 2026-04-11T02:23:51Z

Smoke Test: GitHub Actions Services Connectivity ✅

All connectivity checks passed:

Check	Result
Redis PING (`host.docker.internal:6379`)	✅ `PONG`
PostgreSQL ready (`host.docker.internal:5432`)	✅ accepting connections
PostgreSQL `SELECT 1` (`smoketest` db, user `postgres`)	✅ returned `1`

Note: redis-cli was not pre-installed and sudo is unavailable in this sandbox, so the Redis PING was sent via raw TCP (nc) using the Redis protocol — response was +PONG.

🔌 Service connectivity validated by Smoke Services

github-actions · 2026-04-11T02:24:47Z

🏗️ Build Test Suite Results

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	✅	1/1 passed	✅ PASS
Bun	hono	✅	1/1 passed	✅ PASS
C++	fmt	✅	N/A	✅ PASS
C++	json	✅	N/A	✅ PASS
Deno	oak	N/A	1/1 passed	✅ PASS
Deno	std	N/A	1/1 passed	✅ PASS
.NET	hello-world	✅	N/A	✅ PASS
.NET	json-parse	✅	N/A	✅ PASS
Go	color	✅	1/1 passed	✅ PASS
Go	env	✅	1/1 passed	✅ PASS
Go	uuid	✅	1/1 passed	✅ PASS
Java	gson	✅	1/1 passed	✅ PASS
Java	caffeine	✅	1/1 passed	✅ PASS
Node.js	clsx	✅	All passed	✅ PASS
Node.js	execa	✅	All passed	✅ PASS
Node.js	p-limit	✅	All passed	✅ PASS
Rust	fd	✅	1/1 passed	✅ PASS
Rust	zoxide	✅	1/1 passed	✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Generated by Build Test Suite for issue #1894 · ● 757.3K · ◷

github-actions · 2026-04-11T12:55:34Z

Smoke test report:

PR: perf: optimize secret-digger-claude token usage
PR: refactor: use gh aw logs for token analysis workflows
GitHub MCP merged PR review: ✅
safeinputs-gh PR query: ❌ (tool unavailable)
Playwright GitHub title check: ✅
Tavily web search: ❌ (tool unavailable)
File write/read + bash cat: ✅
Discussion query/comment: ❌ (discussion tools/context unavailable)
npm ci && npm run build: ✅
Overall status: FAIL

🔮 The oracle has spoken through Smoke Codex

lpcox requested a review from Mossaka as a code owner April 11, 2026 02:19

Copilot AI review requested due to automatic review settings April 11, 2026 02:19

Copilot started reviewing on behalf of lpcox April 11, 2026 02:20 View session

Copilot AI reviewed Apr 11, 2026

View reviewed changes

github-actions Bot added the smoke-copilot label Apr 11, 2026

github-actions Bot added the smoke-claude label Apr 11, 2026

This comment has been minimized.

Sign in to view

github-actions Bot added the smoke-services label Apr 11, 2026

github-actions Bot mentioned this pull request Apr 11, 2026

[aw] No-Op Runs #1668

Closed

github-actions Bot added the build-test label Apr 11, 2026

lpcox merged commit 9d06b78 into main Apr 11, 2026
55 of 56 checks passed

lpcox deleted the perf/secret-digger-claude-token-optimization branch April 11, 2026 03:18

github-actions Bot mentioned this pull request Apr 11, 2026

perf: optimize firewall-issue-dispatcher token usage #1904

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

perf: optimize secret-digger-claude token usage#1894

perf: optimize secret-digger-claude token usage#1894
lpcox merged 1 commit intomainfrom
perf/secret-digger-claude-token-optimization

lpcox commented Apr 11, 2026

Uh oh!

github-actions Bot commented Apr 11, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 11, 2026

Uh oh!

Copilot AI Apr 11, 2026

Uh oh!

github-actions Bot commented Apr 11, 2026

Uh oh!

github-actions Bot commented Apr 11, 2026

Uh oh!

This comment has been minimized.

github-actions Bot commented Apr 11, 2026

Uh oh!

github-actions Bot commented Apr 11, 2026

Uh oh!

Uh oh!

github-actions Bot commented Apr 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		BASH_DEFAULT_TIMEOUT_MS: "1800000" # 30 minutes for bash commands
		BASH_MAX_TIMEOUT_MS: "1800000" # 30 minutes max timeout

-                  "Authorization": "$GH_AW_SAFE_OUTPUTS_API_KEY"
+                  "Authorization": "$GH_AW_SAFE_OUTPUTS_API_KEY"
+                },
+                "guard-policies": {
+                  "write-sink": {
+                    "allow": [
+                      "create_issue"
+                    ]
+                  }

Conversation

lpcox commented Apr 11, 2026

Summary

Changes

Expected Impact

Verification

Uh oh!

github-actions Bot commented Apr 11, 2026

✅ Coverage Check Passed

Overall Coverage

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

Copilot AI Apr 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 11, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Apr 11, 2026

🔥 Smoke Test Results — PR #1894

Uh oh!

github-actions Bot commented Apr 11, 2026

Smoke Test Results

Uh oh!

This comment has been minimized.

github-actions Bot commented Apr 11, 2026

Smoke Test: GitHub Actions Services Connectivity ✅

Uh oh!

github-actions Bot commented Apr 11, 2026

🏗️ Build Test Suite Results

Uh oh!

Uh oh!

github-actions Bot commented Apr 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants