Debug agent prompt lacks token usage artifact locations — agents can't find token data without manual guidance

[lpcox]

Problem

When debugging workflow runs, users frequently have to manually instruct agents to look in firewall logs for token usage data. The root cause is that the debug agent prompt (.github/aw/debug-agentic-workflow.md) mentions token usage exists but never tells agents where to find it in the downloaded artifacts.

The documentation itself is comprehensive — docs/src/content/docs/reference/artifacts.md clearly documents that token-usage.jsonl lives in the firewall-audit-logs artifact — but this knowledge is never surfaced in the debug prompt that agents actually follow.

Evidence

What the debug prompt says (~line 273):

"Includes token usage, cost estimates, and execution time"

What the debug prompt does NOT say:

• Which artifact contains token usage data (firewall-audit-logs, not agent)
• The exact file path (api-proxy-logs/token-usage.jsonl)
• How to download it (gh run download <run-id> -n firewall-audit-logs)
• The difference between agent_usage.json (aggregated summary) and token-usage.jsonl (per-request detail)

The common user experience:

1. User asks agent to analyze token usage for a run
2. Agent downloads agent artifact (wrong one for detailed data)
3. Agent finds agent_usage.json with only aggregated totals
4. Agent cannot find per-request token breakdown
5. User manually instructs: "look in the firewall-audit-logs artifact"

Root Causes

1. Fragmented documentation

Token usage information is spread across 5+ files with no cross-references in the debug prompt:

File	What it documents
docs/src/content/docs/reference/artifacts.md	✅ Artifact structure, token-usage.jsonl location, common mistakes
docs/src/content/docs/reference/cost-management.md	✅ gh aw logs token output, cost estimation
docs/src/content/docs/reference/effective-tokens-specification.md	✅ ET computation formula, model multipliers
docs/src/content/docs/reference/audit.md	✅ Token usage breakdown in audit diffs
.github/aw/debug-agentic-workflow.md	❌ Mentions token usage exists but not WHERE

2. Ambiguous "firewall logs" terminology

• "Firewall logs" = network policy logs (allowed/denied domains) — parsed by parse_firewall_logs.cjs
• "Firewall audit logs" = artifact containing token-usage.jsonl — different content entirely
• Users and agents conflate these, leading to confusion

3. Artifact name mismatch

• agent_usage.json (in agent artifact) = aggregated totals only
• token-usage.jsonl (in firewall-audit-logs artifact) = per-request detail
• artifacts.md line 101 explicitly calls this out as a "common mistake" but the debug prompt doesn't link to this

Proposed Solution

1. Add artifact reference section to the debug prompt

In .github/aw/debug-agentic-workflow.md, add a dedicated section after the existing gh aw logs / gh aw audit instructions:

### Token Usage Data Locations

When analyzing token usage from workflow runs:

| Data | Artifact | Path | Content |
|------|----------|------|---------|
| Per-request detail | `firewall-audit-logs` | `api-proxy-logs/token-usage.jsonl` | JSONL with per-API-call token counts, model, effective tokens |
| Aggregated summary | `agent` | `agent_usage.json` | Total input/output/cache tokens, effective tokens |
| Step summary | Job log | "Parse token usage" step | Markdown table in step summary |

**Important:** Token usage detail is in the `firewall-audit-logs` artifact, NOT the `agent` artifact.
To download: `gh run download <run-id> -n firewall-audit-logs`

The `token-usage.jsonl` file contains one JSON object per API request with fields:
- `model`: Model name used
- `input_tokens`, `output_tokens`: Raw token counts  
- `cache_read_tokens`, `cache_write_tokens`: Cache-related tokens
- `effective_tokens`: Cost-normalized token count (see Effective Tokens spec)

2. Add cross-references between documentation files

• artifacts.md → link to cost-management.md for interpretation
• cost-management.md → link to artifacts.md for file locations
• audit.md → link to artifacts.md for underlying data source

3. Clarify firewall terminology in the debug prompt

Add a note distinguishing:

• Firewall network logs (parse_firewall_logs.cjs) — domain allow/deny decisions
• Firewall audit logs artifact (firewall-audit-logs) — contains token usage JSONL, API proxy logs

Files to Modify

1. .github/aw/debug-agentic-workflow.md — Primary fix: add token usage artifact locations and instructions
2. docs/src/content/docs/reference/artifacts.md — Add cross-references to cost-management and audit docs
3. docs/src/content/docs/reference/cost-management.md — Add link to artifacts reference for file locations
4. .github/aw/github-agentic-workflows.md — Consider adding brief "where to find token usage" section

Impact

This will eliminate the need for users to manually guide agents to token usage data during debugging sessions, making gh aw audit and token analysis workflows self-service.