feat: optimize Smoke Services workflow for token efficiency#2214
Conversation
🔥 Smoke Test: Copilot BYOK (Offline Mode)
Running in BYOK offline mode ( Overall: PASS | PR by
|
|
Smoke Test Results: PASS ✅
All smoke tests passed successfully.
|
🔥 Smoke Test Results
PR: feat: optimize Smoke Services workflow for token efficiency Overall: PASS 🟢
|
Smoke TestPRs: chore: upgrade agentic workflows from v0.69.3 to v0.71.1; fix: strip IPv6 http_port listeners when IPv6 is disabled in Squid container Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "registry.npmjs.org"See Network Configuration for more information.
|
There was a problem hiding this comment.
Pull request overview
Optimizes the “Smoke Services” agentic workflow to reduce token/turn usage by removing unused tools, tightening network/host access, and pre-installing required service client binaries outside the agent sandbox.
Changes:
- Removes
agentic-workflowsandcache-memory, and collapses Redis/Postgres checks into a single bash invocation in the workflow instructions. - Adds a runner
steps:pre-install forredis-toolsandpostgresql-client, and reduces timeouts to 10 minutes. - Updates the compiled lock workflow to match the
.mdsource, including switching host access to--allow-host-service-ports 6379,5432and shrinking the allowed domains list.
Show a summary per file
| File | Description |
|---|---|
| .github/workflows/smoke-services.md | Updates workflow config/instructions to reduce tool usage and token footprint; adds runner pre-install step and shorter timeout. |
| .github/workflows/smoke-services.lock.yml | Manually aligns compiled workflow with .md updates (tool removal, host port restrictions, allowed domains, timeouts, pre-install step). |
| .github/aw/actions-lock.json | Removes gh-aw action pin entries (intended as part of workflow/tool simplification), but may impact future determinism of upgrades/compiles. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comments suppressed due to low confidence (1)
.github/aw/actions-lock.json:63
.github/aw/actions-lock.jsonno longer contains pins forgithub/gh-aw-actions/setup@v0.71.1andgithub/gh-aw-actions/setup-cli@v0.71.1, but the repo still uses those actions widely (e.g..github/workflows/agentics-maintenance.ymland many*.lock.ymlfiles). This makes futuregh aw upgrade/gh aw compileruns less reproducible and can force network lookups for those tags, which is especially problematic in firewalled environments. Please restore entries for these actions (repo, version, sha) so action resolution stays deterministic.
"version": "v4.0.0",
"sha": "4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd"
},
"github/gh-aw/actions/setup@v0.69.3": {
"repo": "github/gh-aw/actions/setup",
"version": "v0.69.3",
"sha": "6abd7107cebd8b300f4d64013201ad2d8e0a994f"
},
"softprops/action-gh-release@v3.0.0": {
- Files reviewed: 3/3 changed files
- Comments generated: 1
| @@ -213,7 +213,6 @@ jobs: | |||
| cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" | |||
| cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" | |||
| cat "${RUNNER_TEMP}/gh-aw/prompts/agentic_workflows_guide.md" | |||
There was a problem hiding this comment.
This workflow removes the agentic-workflows tool, but the prompt assembly still includes cat "${RUNNER_TEMP}/gh-aw/prompts/agentic_workflows_guide.md". Keeping this guide increases prompt size (hurting the token-efficiency goal) and can confuse the agent by documenting a tool that is no longer available. Please remove this prompt include (and any related prompt section) now that agentic-workflows is not enabled for this workflow.
| cat "${RUNNER_TEMP}/gh-aw/prompts/agentic_workflows_guide.md" |
🏗️ Build Test Suite Results
Overall: 8/8 ecosystems passed — ✅ PASS
|
* chore: recompile all agentic workflows Recompile all 30 lock files with gh-aw v0.71.1 compiler to pick up changes from recently merged workflow .md updates. Key changes: - Remove sha256 digest pins from container image references - smoke-services.lock.yml updates from #2214 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * chore: upgrade mcpg from v0.2.29 to v0.3.0 in remaining workflows Update sandbox.mcp.version from v0.2.29 to v0.3.0 in: - smoke-claude.md - smoke-codex.md - smoke-copilot.md All 30 workflows now use mcpg v0.3.0. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Smoke Services averaged 173K tokens/run and 14 turns/run — highest turn count of all smoke workflows — largely because the agent was installing
redis-toolsandpostgresql-clientviaapt-geton every run and running each connectivity check as a separate tool call.Changes
smoke-services.mdsteps:—redis-toolsandpostgresql-clientare installed on the runner before the agent starts. AWF bind-mounts/usrread-only into the agent chroot, so the binaries are immediately available inside the sandbox without any agent-side installation.pg_isready,psql SELECT 1) are collapsed into one bash invocation; agent just reads output and posts a comment (~3 turns vs ~14).agentic-workflowstool — unused for a connectivity test; eliminates theInstall gh-aw extensionstep,agenticworkflowsMCP server, andalpine:latestcontainer pull.cache-memorytool — stateless test needs no cross-run memory; drops 7 steps (create dir, TTL key, restore, setup git repo, strip execute bits, scan injection, commit) and theactions/cacheaction.nodefromnetwork.allowed— no npm/Node.js usage; shrinks the domain allowlist by ~24 entries, reducing system prompt size on every turn.timeout-minutes15 → 10 — task is now simple enough that 10 min is generous.smoke-services.lock.ymlManually updated to match all
.mdchanges while preserving the v0.71.1 compiled infrastructure. Notable:--enable-host-access --allow-host-ports 80,443,8080replaced with--allow-host-service-ports 6379,5432(more restrictive — only the two service ports are opened tohost.docker.internal).Expected impact: ~40–50K tokens/run (~3 turns) vs current ~173K/run (~14 turns).
Warning
Firewall rules blocked me from connecting to one or more addresses (expand for details)
I tried to connect to the following addresses, but was blocked by firewall rules:
https://api.github.com/repos/actions/github-script/git/ref/tags/v9/usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq .object.sha(http block)https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v0.68.3/usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.68.3 --jq .object.sha(http block)https://api.github.com/repos/github/gh-aw/releases/latest/usr/bin/gh gh extension install github/gh-aw --pin v0.71.1(http block)/usr/bin/gh gh extension install github/gh-aw(http block)If you need me to access, download, or install something from one of these locations, you can either: