Safe Output Health Report - 2026-03-21

[github-actions[bot]]

Executive Summary

• Period: 2026-03-20 23:00Z – 2026-03-21 05:00Z (last 24 hours)
• Runs Analyzed: 79 (37 completed, 40 skipped, 2 in-progress)
• Unique Workflows Active: 41
• Safe Output Jobs Executed: 75
• Safe Output Jobs Failed: 2 (97.3% success rate)
• Safe Output Operations Executed: 62 across 26 runs
• Error Clusters Identified: 2 (both are recurring known patterns)
• New Patterns: None

Overall Health: 🟡 GOOD with minor recurring issues — Two failures, both matching well-understood patterns EP006 and EP016. No new error types emerged.

---

Safe Output Job Statistics

Job Type	Total Executions	Failures	Success Rate
create_issue	15	0	100%
add_comment	9	1	89%
create_pull_request	7	0	100%
create_discussion	7	0	100%
create_pull_request_review_comment	4	0	100%
submit_pull_request_review	3	0	100%
update_pull_request	2	0	100%
set_issue_type	2	0	100%
send_slack_message	2	0	100%
push_to_pull_request_branch	1	1	0%
other (dispatch_workflow, add_reviewer, add_labels, remove_labels, update_release)	7	0	100%

---

Error Clusters

Cluster 1: EP016 — push_to_pull_request_branch Missing Required Labels

• Count: 1 occurrence
• Affected Jobs: push_to_pull_request_branch
• Affected Workflows: Smoke Update Cross-Repo PR (§23368643312)
• Error Message:

  ✗ Message 2 (push_to_pull_request_branch) failed: Pull request is missing required labels: smoke-test. Current labels:
  ⚠️ Code push operation 'push_to_pull_request_branch' failed — remaining safe outputs will be cancelled
  ⏭ Message 3 (add_comment) cancelled — Cancelled: code push operation failed
  ```
  

• Root Cause: The smoke-update-cross-repo-pr workflow's push_to_pull_request_branch handler requires the target PR (githubnext/gh-aw-side-repo PR rejig docs #1) to have the smoke-test label before applying a patch. When the PR does not have this label, the push is blocked and subsequent safe outputs are cascaded-cancelled.
• Impact: 1 workflow failure, 2 safe outputs cancelled (push + add_comment). Smoke test unable to complete.

Cluster 2: EP006 — add_comment Context Mismatch (Schedule-Triggered)

• Count: 1 occurrence
• Affected Jobs: add_comment
• Affected Workflows: Smoke Multi PR (§23368726596)
• Error Message:

  ✗ Message 3 (add_comment) failed: Target is "triggering" but not running in issue or pull request context, skipping add_comment
  

• Root Cause: The agent in smoke-multi-pr emitted an add_comment with target: "triggering" but the workflow was triggered by a schedule event, not by an issue/PR action. No triggering entity exists to comment on.
• Impact: 1 workflow failure. The two PRs were successfully created; only the result comment was lost.

View Non-Critical Errors (Out of Scope)

Cache Memory Artifact Not Found (8 occurrences)

Eight workflow runs produced Unable to download artifact(s): Artifact not found for name: cache-memory in their update_cache_memory job logs. This is benign — it occurs when a workflow has never successfully uploaded a cache-memory artifact before (first run, or after cache expiry). All affected workflows still completed as success:

• Instructions Janitor (23367184603)
• Smoke Gemini (23368364917)
• Smoke Claude (23368377758, 23368452880)
• Smoke Copilot (23368452920, 23368835222)
• Smoke Update Cross-Repo PR (23368643312)
• Q (23371069812)

smoke-codex Validation Error (2 occurrences — Agent-Level, Out of Scope)

Two runs had ERR_VALIDATION: File workflows/smoke-codex.md contains unauthorized GitHub Actions expressions at the activation job level. This is an agent activation failure, not a safe output failure, and is tracked separately.

---

Root Cause Analysis

EP006: Recurring Schedule-Context Mismatch

EP006 has appeared in at least 5 of the last 10 audit days (2026-02-25, 2026-03-13, 2026-03-16, 2026-03-20, 2026-03-21). The pattern is consistent: a smoke test workflow uses add_comment with target: "triggering" as part of its result reporting, but the workflow is triggered by schedule (not by an issue or PR comment). The safe output handler correctly detects the mismatch and rejects the operation — but this causes the workflow to conclude as failure.

The fix would be either:

1. Having the smoke test workflow use add_comment with an explicit issue_number or pull_request_number target rather than "triggering", or
2. Changing the smoke test to use create_issue or noop for schedule-triggered result reporting.

EP016: Missing Required Label for push_to_pull_request_branch

EP016 has been observed on 2026-03-16, 2026-03-20, and 2026-03-21 (3 of last 6 days). The smoke-update-cross-repo-pr workflow requires githubnext/gh-aw-side-repo PR #1 to carry the smoke-test label before it will apply a patch. If the label is absent (e.g., after it was removed or never applied), the handler rejects the push with a hard error that cascades to cancel remaining safe outputs.

The fix would be either:

1. Adding logic in the smoke workflow or harness to ensure the smoke-test label is applied to the cross-repo PR before attempting the push, or
2. Relaxing the label validation to a warning rather than a hard failure when in a smoke test context.

---

Recommendations

High Priority (Recurring Issues)

1. Fix EP006 - add_comment Context Mismatch in smoke-multi-pr

   • Priority: Medium
   • Root Cause: add_comment with target: "triggering" used in schedule-triggered workflow
   • Recommended Action: Update smoke-multi-pr workflow to use create_issue or explicit PR number targeting for result reporting when running on schedule
   • Affected: Smoke Multi PR workflow

2. Fix EP016 - Ensure smoke-test label on cross-repo PR

   • Priority: Medium
   • Root Cause: push_to_pull_request_branch requires smoke-test label; label missing at execution time
   • Recommended Action: Add a step in smoke-update-cross-repo-pr to apply smoke-test label to the target PR before attempting the patch push; OR add a label-application safe output type
   • Affected: Smoke Update Cross-Repo PR workflow

---

Historical Context

View 7-Day Trend

Date	Runs Analyzed	SO Job Failures	Success Rate	Main Errors
2026-03-15	100	0	100%	Clean
2026-03-16	239	4	81.8%	EP006 x2, EP015 x1, EP016 x1
2026-03-17	13	1	93.3%	GitHub App token 404
2026-03-18	176	1	83.3%	EP018 x1
2026-03-19	120	0	100%	Clean (1 warning)
2026-03-20	98	4	90.2%	EP006 x2, EP015 x1, EP016 x1
2026-03-21	79	2	97.3%	EP016 x1, EP006 x1

• Error rate trend: Improving compared to last two days (2 failures vs 4 on 2026-03-20/03-16)
• Most reliable job type: create_issue, create_discussion, create_pull_request (100% today)
• Most problematic job type: push_to_pull_request_branch (0% today, EP016)
• Recurring pattern: EP006 + EP016 continue to appear intermittently in smoke test workflows

---

Metrics and KPIs

• Overall Safe Output Job Success Rate: 97.3%
• Overall Safe Output Operation Success Rate: ~96.8% (60/62 operations, 2 hard failures)
• Most Reliable Job Type: create_issue (15/15), create_discussion (7/7), create_pull_request (7/7)
• Most Problematic Job Type: push_to_pull_request_branch (0/1 today, EP016)
• Compared to Yesterday: +7.1 percentage points improvement (97.3% vs 90.2%)

---

Next Steps

• Investigate EP016 fix: auto-apply smoke-test label to githubnext/gh-aw-side-repo PR rejig docs #1 before patch push
• Investigate EP006 fix: update smoke-multi-pr to use explicit target for schedule-triggered runs
• Monitor EP006/EP016 frequency — both have now appeared 3+ days in last week; consider escalating priority if they persist through next week

References:

• §23368643312 — Smoke Update Cross-Repo PR (EP016 failure)
• §23368726596 — Smoke Multi PR (EP006 failure)
• §23371777974 — This audit run

AI generated by Safe Output Health Monitor · history

• expires on Mar 22, 2026, 4:23 AM UTC

[github-actions[bot]]

🤖 Beep boop! The smoke test agent has landed! 🛬\n\nJust popping by to say: your Safe Output Health Report is looking chef's kiss 🍕 — 97.3% success rate? That's basically a flying robot doing parkour and sticking the landing. 🦾\n\nSee you in the next test run! 🚀\n\n*(This comment was generated by Smoke Test run §23372042458)*

Note

🔒 Integrity filtering filtered 1 item

Integrity filtering activated and filtered the following item during workflow execution.
This happens when a tool call accesses a resource that does not meet the required integrity or secrecy level of the workflow.

• pr:Extract shared HTTP MCP server lifecycle into mcp_http_server_runner.cjs #22078 (pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)

📰 BREAKING: Report filed by Smoke Copilot · ◷

[github-actions[bot]]

💥 WHOOSH! ZAP! POW! 🦸

The Smoke Test Agent has landed! 🤖✨

— Agent Claude, Smoke Run §23372042485, swooping in and OUT! 💨

Note

🔒 Integrity filtering filtered 1 item

Integrity filtering activated and filtered the following item during workflow execution.
This happens when a tool call accesses a resource that does not meet the required integrity or secrecy level of the workflow.

• pr:Extract shared HTTP MCP server lifecycle into mcp_http_server_runner.cjs #22078 (pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)

💥 [THE END] — Illustrated by Smoke Claude · ◷

[github-actions[bot]]

This discussion has been marked as outdated by Safe Output Health Monitor.

A newer discussion is available at Discussion #22230.