feat: add copilot-token-audit and rewrite copilot-token-optimizer

[lpcox]

Summary

Replace the old copilot-token-usage-analyzer with two focused workflows:

New: copilot-token-audit

Daily audit that tracks Copilot token consumption across all workflows:

• Downloads 30 days of logs via gh aw logs --json (installed via gh extension install)
• Computes per-workflow metrics (tokens, cost, turns, action minutes, errors)
• Persists daily snapshots to memory/token-audit repo-memory branch
• Maintains a rolling 90-day summary for trend analysis
• Generates charts (top workflows bar chart, historical trend line)
• Publishes audit discussion via shared/daily-audit-discussion.md

Rewritten: copilot-token-optimizer

Daily optimizer that digs into a single high-usage workflow:

• Reads audit snapshots from repo-memory to select the heaviest consumer
• Tracks optimization history to avoid re-analyzing recently targeted workflows
• Uses agentic-workflows MCP tools (logs, audit) for deep per-run analysis
• Downloads firewall data via gh aw logs --firewall
• Analyzes tool usage vs configured tools, cache efficiency, error patterns, prompt size
• Produces conservative recommendations backed by evidence from 5+ runs
• Publishes optimization discussion with tool usage matrix and estimated savings

Removed: copilot-token-usage-analyzer

Replaced by copilot-token-audit above.

Key design decisions

• No ./gh-aw (built from source) — uses gh extension install github/gh-aw exclusively
• Shared repo-memory branch — both workflows read/write memory/token-audit; audit writes snapshots, optimizer writes optimization-log.json
• Conservative optimizer — checks 5+ runs before recommending tool removal, never removes tools used in any observed run
• Leverages shared components — daily-audit-discussion, repo-memory-standard, reporting, python-dataviz

[lpcox]

@copilot resolve the merge conflicts in this pull request. prioritize updates from this PR

[Copilot]

Pull request overview

This PR replaces the legacy Copilot token usage analyzer workflow with two more focused daily workflows: a token audit that records historical snapshots and an optimizer that targets one high-usage workflow per day using deeper run-level analysis.

Changes:

• Removed copilot-token-usage-analyzer workflow.
• Added copilot-token-audit workflow + lock file to collect 30-day Copilot usage, persist snapshots to repo-memory, and publish an audit discussion with charts.
• Rewrote copilot-token-optimizer workflow + lock file to select a heavy consumer from repo-memory snapshots and publish evidence-based optimization recommendations.

Show a summary per file

File	Description
.github/workflows/copilot-token-usage-analyzer.md	Removed the old analyzer workflow definition.
.github/workflows/copilot-token-audit.md	New audit workflow prompt/spec for collecting and persisting token usage snapshots + charts.
.github/workflows/copilot-token-audit.lock.yml	Compiled workflow for the new audit, including repo-memory push and discussion publishing.
.github/workflows/copilot-token-optimizer.md	Rewritten optimizer prompt/spec to consume audit snapshots and perform deeper per-run analysis.
.github/workflows/copilot-token-optimizer.lock.yml	Compiled workflow for the rewritten optimizer, including repo-memory push and discussion publishing.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (4)

.github/workflows/copilot-token-optimizer.md:92

• Same repo-memory path issue for the optimization history: optimization-log.json should live under /tmp/gh-aw/repo-memory/default/memory/token-audit/ (and be created with mkdir -p for that directory). Otherwise the push_repo_memory job’s file glob (memory/token-audit/*.json) won’t include it, so history won’t persist.

# Check if optimization log exists
OPT_LOG="/tmp/gh-aw/repo-memory/default/optimization-log.json"
if [ -f "$OPT_LOG" ]; then

.github/workflows/copilot-token-optimizer.md:122

• gh aw logs supports scoping to a specific workflow via the positional [workflow] argument (gh aw logs [workflow] ...). This command downloads runs across all workflows, which conflicts with the intent (“selected workflow”) and will skew the downstream analysis (and .runs[0].workflow_name may not be the target). Pass the selected workflow name/id as the first argument (and/or filter further if needed).

# Download last 7 days of runs for the selected workflow, with firewall data
gh aw logs \
  --engine copilot \
  --start-date -7d \
  --json \
  --firewall \
  -c 20 \
  > /tmp/gh-aw/token-audit/target-runs.json

.github/workflows/copilot-token-optimizer.md:131

• This jq object construction is invalid because | add isn’t parenthesized inside the object (e.g. total_tokens: [.runs[].token_usage // 0] | add will pipe the whole object into add). Wrap the sums in parentheses, e.g. total_tokens: ([.runs[].token_usage // 0] | add) (and similarly anywhere else you use | inside an object field).

jq '{
  workflow: .runs[0].workflow_name,
  total_runs: (.runs | length),
  total_tokens: [.runs[].token_usage // 0] | add,
  avg_tokens: ([.runs[].token_usage // 0] | add) / ([.runs[].token_usage // 0] | length),
  tool_usage: .tool_usage
}' /tmp/gh-aw/token-audit/target-runs.json

.github/workflows/copilot-token-optimizer.md:82

• If no snapshot is found, $LATEST is empty but the snippet still runs cat "$LATEST" | jq ..., which will error and can derail the run (especially under set -e). Guard the cat/jq with a check (or only run it when $LATEST is non-empty).

  echo "⚠️ No audit snapshots found. The copilot-token-audit workflow may not have run yet."
  echo "Falling back to live data collection..."
fi
echo "Latest snapshot: $LATEST"
cat "$LATEST" | jq '.workflows[:10]'

• Files reviewed: 5/5 changed files
• Comments generated: 2

[Copilot]

Repo-memory paths don’t match the shared/repo-memory-standard.md configuration. With branch-name: "memory/token-audit", snapshots should be written under /tmp/gh-aw/repo-memory/default/memory/token-audit/ (to satisfy the allowed-file patterns used by push_repo_memory). Writing to /tmp/gh-aw/repo-memory/default/YYYY-MM-DD.json / rolling-summary.json will not be pushed and may fail validation.

Suggested change

	2. Copy it to `/tmp/gh-aw/repo-memory/default/YYYY-MM-DD.json` (today's UTC date).
	3. This file is what the optimizer workflow reads to identify high-usage workflows.
	Also maintain a rolling summary file at `/tmp/gh-aw/repo-memory/default/rolling-summary.json` that contains an array of daily overall totals (date, total_tokens, total_cost, total_runs, total_action_minutes) for the last 90 entries. Load the existing file, append today's entry, trim to 90, and save.
	## Phase 3 — Generate Charts
	Create a Python script to generate two charts:
	1. **Token usage by workflow** (horizontal bar chart): Top 15 workflows by total token usage.
	2. **Historical trend** (line chart): Daily total tokens and cost from `rolling-summary.json` — if available. If only 1 data point, skip this chart.
	2. Copy it to `/tmp/gh-aw/repo-memory/default/memory/token-audit/YYYY-MM-DD.json` (today's UTC date).
	3. This file is what the optimizer workflow reads to identify high-usage workflows.
	Also maintain a rolling summary file at `/tmp/gh-aw/repo-memory/default/memory/token-audit/rolling-summary.json` that contains an array of daily overall totals (date, total_tokens, total_cost, total_runs, total_action_minutes) for the last 90 entries. Load the existing file, append today's entry, trim to 90, and save.
	## Phase 3 — Generate Charts
	Create a Python script to generate two charts:
	1. **Token usage by workflow** (horizontal bar chart): Top 15 workflows by total token usage.
	2. **Historical trend** (line chart): Daily total tokens and cost from `memory/token-audit/rolling-summary.json` — if available. If only 1 data point, skip this chart.

[Copilot]

These repo-memory paths should include the memory/token-audit/ subdirectory (e.g. /tmp/gh-aw/repo-memory/default/memory/token-audit/*.json) to match the repo-memory file-glob/constraints from shared/repo-memory-standard.md. As written, the optimizer won’t find the audit snapshots produced by copilot-token-audit, and any writes may be rejected by the repo-memory push step.

This issue also appears in the following locations of the same file:

• line 78
• line 90
• line 115
• line 125

Suggested change

	LATEST=$(ls -1 /tmp/gh-aw/repo-memory/default/*.json 2>/dev/null | grep -v rolling | grep -v optimization | sort -r | head -1)
	LATEST=$(ls -1 /tmp/gh-aw/repo-memory/default/memory/token-audit/*.json 2>/dev/null | grep -v rolling | grep -v optimization | sort -r | head -1)

[Copilot]

@copilot resolve the merge conflicts in this pull request. prioritize updates from this PR

Merge conflicts resolved in e41dad0. The conflicts were from main's removal of old token workflow files — kept the PR's new copilot-token-audit and rewritten copilot-token-optimizer as requested.