Agent Performance Report — 2026-04-21

[github-actions[bot]]

Executive Summary

• Workflows analyzed: 25 (197 total in ecosystem)
• Runs reviewed: 31 (18 completed, 13 in-progress at capture time)
• Average quality score: 72/100 (→ stable, ↓1 from yesterday)
• Average effectiveness score: 68/100 (↓2 from yesterday)
• Top performers: [aw] Failure Investigator, Smoke CI, Test Quality Sentinel, CLI Version Checker
• Needs improvement: Documentation Unbloat, Agent Persona Explorer, AI Moderator
• P0 active: Codex 401 auth — day 4 ([aw-failures] Codex engine 401 auth failure — OPENAI_API_KEY credential missing or invalid #27127, escalation recommended)
• New P1: Design Decision Gate max_turns=5 structurally insufficient for ADR path ([aw-failures] Design Decision Gate: increase max-turns for ADR generation path #27470)

---

Performance Rankings

Top Performing Agents 🏆

1. [aw] Failure Investigator (6h) (Q:90 E:88)

• Two high-quality investigation reports today: [aw-failures] Failure Investigation Report — 2026-04-21 (6h window) #27469 (Design Decision Gate max_turns root-cause with turn-by-turn breakdown) and [aw-failures] AI Moderator (Codex): chatgpt.com blocked by firewall #27412 (AI Moderator chatgpt.com firewall block)
• Correct issue grouping, closed stale [aw] Design Decision Gate 🏗️ failed #27405, clear fix roadmaps
• Best agentic investigator in the ecosystem — consistently delivers actionable, precise outputs

2. Smoke CI (Q:87 E:90)

• 4 runs: 2 success (push to main), 2 cancelled (PR branch race) — correct behavior
• Fast (1.9m–2.7m), zero agentic flags, zero errors on normal runs
• Example: §24704281106 ✅

3. Test Quality Sentinel (Q:82 E:85)

• 4 turns, 3.6min, no agentic flags — clean execution with no quality issues

4. CLI Version Checker (Q:78 E:80)

• Successfully updated 3 CLI tools today: Claude Code 2.1.116, Copilot CLI 1.0.34, Codex 0.122.0 ([ca] Update CLI versions: Claude Code 2.1.116, Copilot 1.0.34, Codex 0.122.0 #27484)
• ⚠️ Flagged resource_heavy (26 turns, $0.82) and partially_reducible — version fetching should move to deterministic pre-agent steps

5. Design Decision Gate 🏗️ (Q:78 E:78)

• Today's run succeeded (4 turns, $0.37, partially_reducible)
• ⚠️ Structural issue identified: max_turns=5 is insufficient for the ADR write+push path (needs 6+). Fix tracked in [aw-failures] Design Decision Gate: increase max-turns for ADR generation path #27470

6. Issue Monster (Q:75 E:80)

• 2 successful runs, 3 turns each, read-only posture
• Both flagged overkill_for_agentic (2nd consecutive day) — candidate for deterministic replacement

Agents Needing Improvement 📉

Documentation Unbloat (Q:48 E:52)

• 58 turns, $2.46, 14.8min — most expensive daily workflow
• 0 safe outputs produced despite high resource consumption
• Flags: resource_heavy + partially_reducible (50% data-gathering turns are wasteable)
• Recommendation: add pre-agent deterministic file listing; require ≥1 safe output or fail fast; cap at max_turns: 30

Agent Persona Explorer (Q:52 E:58)

• 42 turns, 2.2M tokens, resource_heavy + poor_agentic_control + partially_reducible (95%!) + model_downgrade
• Only 1 safe output from 95% data-gathering turns
• Recommendation: move persona data fetching to pre-agent shell steps; downgrade to claude-haiku-4-5

AI Moderator (Q:10 E:5) 🚨

• Day 4 of Codex 401 auth ([aw-failures] Codex engine 401 auth failure — OPENAI_API_KEY credential missing or invalid #27127) — escalation recommended (no resolution in 4 days)
• New failure mode discovered: chatgpt.com blocked by firewall during Codex execution ([aw-failures] AI Moderator (Codex): chatgpt.com blocked by firewall #27412) — separate issue
• Recommendation: switch engine to copilot or claude as temporary fallback while Codex P0 is unresolved

GitHub Remote MCP Auth Test (Q:40 E:0)

• Persistent auth failure; 3 turns, 1 error — no improvement observed

---

Quality Analysis

Quality Distribution & Assessment Flags

Quality Tier	Count	Workflows
Excellent (80-100)	4	Failure Investigator, Smoke CI, Test Quality Sentinel, Issue Monster
Good (60-79)	2	CLI Version Checker, Design Decision Gate
Fair (40-59)	2	Documentation Unbloat, Agent Persona Explorer
Poor (<40)	2	AI Moderator, GitHub Remote MCP Auth Test

Assessment flags today:

• resource_heavy_for_domain (high severity): Documentation Unbloat (58T), Agent Persona Explorer (42T), CLI Version Checker (26T)
• partially_reducible (low-medium): 4 workflows — data-gathering turns should move to deterministic steps
• poor_agentic_control (medium): Agent Persona Explorer — broad, weakly controlled behavior
• overkill_for_agentic (low): Issue Monster (both runs)
• model_downgrade_available (low): Agent Persona Explorer

---

Behavioral Patterns

Productive ✅

• Failure Investigator → Issue creation: Excellent root-cause investigation → precise, actionable issues with fix roadmaps
• CLI Version Checker → PR: Reliable daily automation producing clear version bump PRs
• Smoke CI → baseline: Consistent fast CI health check

Problematic ⚠️

• 95% data-gathering in agentic turns (Agent Persona Explorer): Near-total agentic time spent on data collection, not decision-making
• No-output expensive runs (Documentation Unbloat): $2.46/run, 0 tangible outputs — clearest cost drain in ecosystem
• Overkill agentic use (Issue Monster): Flagged stable enough for deterministic automation 2 days in a row
• Multi-mode failure (AI Moderator): Two independent failure mechanisms now active (401 auth + chatgpt.com firewall) — needs P0 escalation

---

Recommendations

High Priority

1. Fix Design Decision Gate max_turns → [aw-failures] Design Decision Gate: increase max-turns for ADR generation path #27470 (tracked)

   • Increase from 5 → 7 OR move mkdir docs/adr to pre-agent step
   • Effort: ~15 min | Impact: eliminates structural ADR path failure

2. Add outcome gate to Documentation Unbloat

   • Require ≥1 safe output per run or exit early; add deterministic pre-agent file listing; cap at max_turns=30
   • Effort: 1-2 hours | Impact: reduce $2.46/run cost by 50%+

3. Optimize Agent Persona Explorer

   • Move 95% data-gathering to pre-agent steps; downgrade to claude-haiku-4-5
   • Target: ≤10 turns, <$0.30/run | Effort: 2-3 hours

4. AI Moderator temp engine fallback

   • Switch to copilot while Codex P0 persists (day 4, no resolution)
   • Effort: 5 min | Impact: restore AI Moderator function immediately

Medium Priority

5. CLI Version Checker: Add deterministic pre-agent version fetch (target ≤10 turns, <$0.30/run)
6. Issue Monster: Evaluate deterministic replacement (flagged overkill 2+ consecutive days)
7. Copilot P0 auth: Escalate Codex 401 — day 4 with no resolution is ecosystem-blocking

---

Trends

Metric	Apr 19	Apr 20	Apr 21	Trend
Quality score	76	73	72	↓ declining
Effectiveness	75	70	68	↓ declining
Codex P0	Day 1	Day 3	Day 4	🔴 unresolved
resource_heavy flags	~8	11	3 (today's window)
Documentation Unbloat cost	~$2.40	~$2.40	$2.46	→ flat

Positive signals today:

• CLI tools successfully updated (3 tools via [ca] Update CLI versions: Claude Code 2.1.116, Copilot 1.0.34, Codex 0.122.0 #27484) ✅
• Failure Investigator produced highest-quality reports this week ✅
• Design Decision Gate structural issue discovered and tracked before further failures ✅

---

Actions This Run

• Created this performance report discussion
• Analyzed 31 runs across 25 workflows with agentic assessment data
• Updated shared memory: agent-performance-latest.md + shared-alerts.md
• No new improvement issues needed (all active patterns already tracked in open issues)

References:

• §24704272587 (this run)
• §24704281106 (Smoke CI success today)
• §24685186822 (Failure Investigator, Apr 21)

Generated by Agent Performance Analyzer - Meta-Orchestrator · ● 1.3M · ◷

• expires on Apr 22, 2026, 4:47 AM UTC

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-04-22T04:47:32.419Z.

Closed by Workflow