Workflow Health Dashboard — 2026-03-10 #20316
Description
Overview
This issue tracks the operational health of all 166 agentic workflows in this repository as of 2026-03-10.
Summary
| Category | Count | % |
|---|---|---|
| ✅ Healthy | 154 | 93% |
| 5 | 3% | |
| ❌ Critical | 7 | 4% |
| 🔇 Inactive | 0 | 0% |
Overall Health Score: 70/100 (↓2 from 72 — ongoing infrastructure failures)
Critical Issues 🚨
P1: Lockdown Token Missing — 3 Workflows
- Issue Monster (~50+ failures/day every 30 min) — no
GH_AW_GITHUB_TOKEN - PR Triage Agent (~4 failures/day every 6h) — same lockdown error
- Daily Issues Report (1 failure/day) — same lockdown error
- Action: Tracking issue created — [P1] Lockdown token failures: Issue Monster, PR Triage Agent, Daily Issues Report #20315
- Root cause:
GH_AW_GITHUB_TOKENsecret not provisioned in repository
P1: OpenAI/Codex Engine Failures — 2 Workflows
- Smoke Codex — "Execute Codex" step failing — Issue #20285 (expires Mar 17)
- Duplicate Code Detector — "Execute Codex" step failing — Issue #20304 (expires Mar 17)
- Pattern: Both Codex-engine workflows failing consistently since Mar 9
Warnings ⚠️
P2 Issues with Tracking (4 workflows)
| Workflow | Issue | Error | Expires |
|---|---|---|---|
| Safe Output Health Monitor | #20305 | "Download logs" step failure | Mar 17 |
| Smoke Update Cross-Repo PR | #20288 | Pre-agent failure | Mar 17 |
| Smoke Gemini | New — safe_outputs add_comment context error |
Scheduled trigger has no triggering issue/PR | N/A |
| Smoke Multi PR | Recent failure (2026-03-10T01:13Z) | Not yet analyzed | N/A |
P2 Issues without Tracking (3 workflows)
- Weekly Safe Outputs Spec Review — failed 2026-03-09T22:47Z — no tracking issue
- Org Health Report — periodic failures due to lockdown (same root cause as P1)
- Contribution Check — 1 failure detected today
Compilation Status ✅
- 166/166 workflows compiled successfully
- 0 missing lock files
- 13 workflows appear to have stale lock files but this is a false positive — same-timestamp git checkout artifact
Systemic Issues
GH_AW_GITHUB_TOKEN Missing (Critical)
- Affected: Issue Monster, PR Triage Agent, Daily Issues Report, Org Health Report
- Pattern: All use
lockdown: truewhich requires a custom GitHub token - All programmatic fix paths have been closed — requires admin intervention
- Impact: Issue tracking, PR triage, and daily reporting all degraded
Codex Engine Failures (High)
- Affected: Smoke Codex, Duplicate Code Detector
- Pattern: Consistent since 2026-03-09
- Impact: OpenAI cybersec restrictions may be blocking execution
Healthy Workflows ✅
154 workflows operating normally, including:
- Smoke Copilot ✅ | Smoke Claude ✅ | Smoke Create Cross-Repo PR ✅
- Metrics Collector ✅ | Agentic Maintenance ✅ | Chroma Issue Indexer ✅
- Auto-Triage Issues ✅ | Bot Detection ✅ | jsweep ✅
Trends (7-Day)
| Date | Score | Failures | Notes |
|---|---|---|---|
| Mar 1 | 73/100 | 3 P1 | Lockdown started |
| Mar 3 | 76/100 | 5 P1 | AI Moderator added |
| Mar 7 | 74/100 | 6 P1+P2 | Codex issues begin |
| Mar 9 | 72/100 | 8+ | Multiple new failures |
| Mar 10 | 70/100 | 10+ | Codex + lockdown + smoke |
Recommendations
- URGENT: Provision
GH_AW_GITHUB_TOKENsecret — 4+ workflows blocked daily - HIGH: Investigate Codex engine failures ([aw] Smoke Codex failed #20285, [aw] Duplicate Code Detector failed #20304) — may need API key rotation
- MEDIUM: Fix Smoke Gemini
add_commentcontext targeting (usescheduleevent guard) - LOW: Investigate Smoke Multi PR and Weekly Safe Outputs failures
Last updated: 2026-03-10T07:27:00Z
Run: §22891709570
Next check: 2026-03-11T07:00Z
Generated by Workflow Health Manager - Meta-Orchestrator · ◷
- expires on Mar 11, 2026, 7:36 AM UTC