[copilot-token-optimizer] Token Optimization: Contribution Check — Deep Research & Toolset Reduction

[github-actions]

🔍 Optimization Target: Contribution Check

Selected because: Highest-token workflow not optimized in the last 14 days (5.7M tokens on 2026-04-13)
Analysis period: 2026-04-04 to 2026-04-13 (7 daily snapshots, 2 runs with full telemetry)
Runs analyzed: 2 runs with detailed token data (both from 2026-04-13)

---

📊 Token Usage Profile

Metric	Value
Total tokens (today, 2026-04-13)	5,734,850
Peak day tokens (2026-04-04)	11,248,952
Avg tokens/run (active runs)	~2,813,000
Avg effective tokens/run	~3,200,000
Avg turns/run	91
Cache efficiency	~45.4%
Input : Output ratio	~53 : 1
Agentic fraction	50% (data-gathering)

Both audited runs were flagged with resource_heavy_for_domain (high severity) and partially_reducible (50% of turns are data-gathering that could move to deterministic steps).

---

🔧 Recommendations

1. Trim the "Deep Research" step in the subagent — Est. savings: ~1,100,000 tokens/run

Evidence: The subagent's Step 2.5 instructs it to do five types of expensive exploration per PR:

1. Browse repo directory structure + README + architecture docs
2. Read surrounding code for every changed file (not just the diff)
3. Read any referenced issues
4. Map existing test patterns and frameworks
5. Search for open PRs touching the same files

With up to 10 PRs dispatched per batch and ~9 turns/PR observed, the subagent is spending roughly 5–6 of those turns on deep research that goes well beyond what the checklist questions require. The checklist only needs:

• CONTRIBUTING.md content
• PR title, body, author, labels
• Changed file paths + diff

Steps 1, 3, 4, and 5 of the deep research are not gated by any checklist question. Step 2 (surrounding context) could be reduced to reading only directly-changed files rather than exploring the whole module.

Action: Simplify Step 2.5 in .github/agents/contribution-checker.agent.md to:

## Step 2.5: Targeted Context

Before running the checklist, gather targeted context:
- Read the PR diff and changed files carefully to understand what's changing.
- If the PR body references an issue number, read that issue to understand the original requirements.

Do not browse the repo directory, read surrounding code, or search for duplicate PRs.
This focused approach gives you enough context for a high-quality checklist without expensive exploration.

Estimated turn reduction: from ~9 turns/PR to ~4 turns/PR → ~50 fewer turns/run → ~1.1M effective tokens saved.

---

2. Pre-fetch CONTRIBUTING.md once in the pre-agent step — Est. savings: ~120,000 tokens/run

Evidence: The subagent fetches CONTRIBUTING.md as its very first action (Step 1). When 10 PRs are dispatched, this document is fetched independently by each of the 10 subagent instances. The file content is identical across all calls.

Action: Add a step to contribution-check.md's frontmatter bash section to read the file once and write it to $GITHUB_WORKSPACE/contributing-guidelines.md. Then pass its content inline in the subagent dispatch prompt:

# In the pre-agent step (or a new step)
gh api repos/$TARGET_REPOSITORY/contents/CONTRIBUTING.md \
  --jq '.content' | base64 -d > "$GITHUB_WORKSPACE/contributing-guidelines.md" 2>/dev/null \
  || echo "# No CONTRIBUTING.md found" > "$GITHUB_WORKSPACE/contributing-guidelines.md"
```

Update the subagent prompt to start with:
```
The CONTRIBUTING.md content for this repository is attached below.
Skip Step 1 — do not fetch CONTRIBUTING.md again.

This eliminates 9 redundant file fetches (~15K tokens each) and removes one tool call per subagent turn.

---

3. Narrow the GitHub toolset — Est. savings: ~80,000 tokens/run

Evidence: The workflow configures toolsets: [default], which exposes the full GitHub toolset (typically 30+ tools). The subagent only requires:

• pull_requests — to read PR metadata, diff, and changed files
• repos — to read CONTRIBUTING.md and repo file contents

The tool schemas for unused tools (org management, Actions runs, releases, projects, code scanning, etc.) are included in the context for every single turn, consuming input tokens on all 89–93 requests.

Action: Change the tools configuration in .github/workflows/contribution-check.md:

tools:
  github:
    toolsets: [pull_requests, repos]
    allowed-repos: all
    min-integrity: none

Note: If issues lookup (for PR-referenced issues) is required, add issues to the toolset. Review one or two runs after the change to confirm no missing-tool warnings appear.

---

4. Reduce max PRs per batch from 10 to 5 — Est. savings: ~1,400,000 tokens/run (on busy days)

Evidence: Token usage on 2026-04-04 hit 11.2M tokens — roughly 4× the baseline run cost. This is consistent with a run that hit the 10-PR cap. On moderate days the cost drops to ~3M (3 PRs evaluated).

Halving the batch cap to 5 PRs would cut token usage roughly proportionally on high-traffic days while ensuring each PR receives equally thorough evaluation. Skipped PRs are already tracked in the report (skipped_count) and can be caught in the next scheduled run (every 4 hours).

Action: In the Fetch and filter PRs step in .github/workflows/contribution-check.md:

MAX_EVALUATE=5   # Reduced from 10

This is a conservative trade-off: maintainers still see results within 8 hours for any PR, and cost on busy days drops from ~11M to ~5.5M tokens.

---

📉 Combined Savings Estimate

Recommendation	Est. savings/run	Confidence
Trim Deep Research	~1,100,000 tokens	High
Pre-fetch CONTRIBUTING.md	~120,000 tokens	High
Narrow toolset	~80,000 tokens	Medium
Reduce max PRs to 5	~1,400,000 on busy days	Medium
Total (conservative, recs 1–3)	~1,300,000 tokens/run	High

Applied together, recommendations 1–3 could reduce an average active run from ~2.8M to ~1.5M tokens — roughly 46% reduction.

---

Tool Usage Matrix

Tool	Configured?	Needed?	Used in N/2 runs	Recommendation
pull_request_read (get, get_diff, get_files)	✅ via default	✅ Yes	2/2	Keep
get_file_contents (CONTRIBUTING.md)	✅ via default	✅ Yes	2/2	Keep (optimize to pre-fetch)
issue_read	✅ via default	⚠️ Conditional	Unknown	Keep (used if PR refs an issue)
list_pull_requests	✅ via default	❌ Only for duplicate search	Unknown	Remove if Deep Research trimmed
search_code	✅ via default	❌ Only for duplicate search	Unknown	Remove if Deep Research trimmed
Actions, releases, orgs, projects	✅ via default	❌ No	0/2	Remove — narrow toolset

Audited Runs Detail

Run	Tokens	Effective Tokens	Turns	Duration	Cache Efficiency	Conclusion
§24345466010	2,794,118	3,172,755	93	8.9 min	45.1%	✅ success
§24335760500	2,833,134	3,227,269	89	11.2 min	45.8%	✅ success

Both runs used claude-sonnet-4.6 exclusively. No errors, no missing tools, no missing data.

Agentic assessments (both runs):

• resource_heavy_for_domain (HIGH): General automation task consuming heavy execution profile
• partially_reducible (LOW): ~50% of turns are data-gathering that could move to deterministic steps

---

⚠️ Caveats

• Recommendations are based on 2 audited runs — both from the same day with the same codebase state
• Some Deep Research steps may occasionally provide value (e.g., catching issues in complex refactors). Consider A/B testing with a subset of runs before fully removing Step 2.5
• Narrowing the toolset (Rec 3) should be verified: if any missing_tool warnings appear after the change, re-add the issues toolset
• The MAX_EVALUATE=5 change (Rec 4) trades coverage for cost — appropriate if the 4-hour schedule provides sufficient catch-up time

References:

• §24345466010 — Run 373 (Apr 13 13:17 UTC)
• §24335760500 — Run 372 (Apr 13 09:22 UTC)

Generated by Copilot Token Usage Optimizer · ● 810.5K · ◷

• expires on Apr 20, 2026, 3:15 PM UTC