Safe Output Health Report - 2026-03-22

[github-actions[bot]]

Executive Summary

• Period: 2026-03-22 (last 24 hours)
• Runs Analyzed: 110 (downloaded from 173 total in logs directory)
• Workflows Active: 27 unique workflows
• Safe Output Jobs Executed: 30 (78 skipped — no agent output)
• Safe Output Jobs Failed: 2
• Safe Output Job Success Rate: 93.8%
• Error Clusters Identified: 2 (1 recurring, 1 new variant)

All failures were isolated to smoke test workflows. No production or user-triggered workflow safe output jobs failed.

---

Safe Output Job Statistics

Conclusion	Count	Rate
success	28	93.8%
failure	2	6.7%
skipped (no agent output)	78	—

Workflow	Run	Failed Operation	Pattern
Smoke Update Cross-Repo PR	§23392658504	push_to_pull_request_branch	EP022 (new)
Smoke Multi PR	§23392726144	add_comment	EP006 (recurring)

---

Error Clusters

Cluster 1: push_to_pull_request_branch — Target Branch Missing (EP022 NEW)

• Count: 1 occurrence
• Affected Job: safe_outputs → push_to_pull_request_branch handler
• Affected Workflow: Smoke Update Cross-Repo PR
• Run: §23392658504
• Cascading Effect: add_comment was automatically cancelled (1 additional message lost)

Error message:

fatal: couldn't find remote ref pr-branch
##[error]✗ Message 2 (push_to_pull_request_branch) failed: Failed to fetch branch pr-branch:
The process '/usr/bin/git' failed with exit code 128
##[warning]⚠️ Code push operation 'push_to_pull_request_branch' failed — remaining safe outputs will be cancelled
⏭ Message 3 (add_comment) cancelled — Cancelled: code push operation failed
```

**Root Cause**: The smoke test targets a branch named `pr-branch` in a cross-repo side repository (`githubnext/gh-aw-side-repo`). This branch no longer exists on the remote. The branch may have been deleted after a merged PR or a previous cleanup step. Note: this is distinct from EP016 (PR title prefix validation) — today's failure occurs earlier, at the git fetch phase.

**Impact**: The smoke test failed to validate push-to-PR-branch functionality. Issue #22217 was still created as the first message succeeded before the code push.

---

#### Cluster 2: add_comment Context Mismatch — No Issue/PR Context (EP006 RECURRING)

- **Count**: 1 occurrence (8th+ overall occurrence, recurring since 2026-02-25)
- **Affected Job**: `safe_outputs` → `add_comment` handler
- **Affected Workflow**: Smoke Multi PR
- **Run**: [§23392726144](https://github.com/github/gh-aw/actions/runs/23392726144)
- **Trigger**: `workflow_dispatch`

**Error message:**
```
##[warning]Target is "triggering" but not running in issue or pull request context, skipping add_comment
##[error]✗ Message 3 (add_comment) failed: Target is "triggering" but not running in issue or pull request context

Root Cause: The agent produced add_comment with target=triggering after creating a PR, but the workflow was triggered by workflow_dispatch (not from within an issue or PR). The handler correctly refuses to comment on a non-existent triggering context, but marks the message as failed rather than gracefully skipping it.

Historical Pattern: This error recurs specifically for Smoke Multi PR and Agent Container Smoke Test workflows on workflow_dispatch/schedule triggers. Previous occurrences: 2026-02-25 (2x), 2026-03-16 (2x), 2026-03-20 (2x), 2026-03-21 (1x), 2026-03-22 (1x). The Smoke Multi PR itself succeeded — PRs #22219 and #22220 were created successfully.

---

Root Cause Analysis

Code Push Failures

The push_to_pull_request_branch handler requires an existing remote branch to fetch before pushing. When the target branch is deleted (e.g., after a merged or closed PR), the git fetch fails immediately with exit code 128. The handler has no pre-flight check to verify branch existence before attempting the fetch.

This is a smoke test infrastructure issue — the side-repo PR branch needs to be maintained/recreated between smoke test runs.

Context Mismatch for add_comment

The add_comment handler's target=triggering mode is valid only for issue/PR-triggered runs. The Smoke Multi PR workflow is designed to run both on push events and on workflow_dispatch. When the agent produces add_comment for a dispatch-triggered run, there is no triggering entity. The agent prompt does not have enough context awareness to distinguish between these trigger types.

---

Recommendations

Immediate Actions

1. Recreate pr-branch in githubnext/gh-aw-side-repo (Critical)

   • The Smoke Update Cross-Repo PR workflow requires this branch to exist at all times
   • Without it, every run will fail with exit code 128
   • Action: Create a PR from a branch named pr-branch in the side-repo, or update the smoke test to dynamically create/reset the branch

2. Investigate EP006 recurrence pattern (Medium)

   • EP006 has now occurred on 6 separate days across 2+ months
   • The smoke test workflow should either conditionally skip add_comment on dispatch runs, or the agent prompt should explicitly state: "Do not use add_comment with target=triggering when triggered by workflow_dispatch"

Bug Fixes Required

1. EP022: Add branch existence pre-flight to push_to_pull_request_branch

   • Before attempting git fetch, verify the branch exists via GitHub API (GET /repos/{owner}/{repo}/git/ref/heads/{branch})
   • Provide a more actionable error message explaining why the branch is missing and how to remediate (e.g., "Branch 'pr-branch' not found — ensure the target PR/branch exists before running this workflow")
   • Estimated Effort: Small

2. EP006: Improve context-awareness in agent prompts for schedule/dispatch workflows

   • Add explicit guidance in Smoke Multi PR prompt: when producing add_comment after creating a PR, reference the newly-created PR number (e.g., item_number: created_pr_number) instead of target=triggering
   • Estimated Effort: Small (prompt change only)

Work Item Plans

Work Item 1: Fix Smoke Update Cross-Repo PR side-repo branch (Critical)

• Type: Infrastructure Fix
• Priority: Critical
• Description: The pr-branch branch in githubnext/gh-aw-side-repo is missing, causing all Smoke Update Cross-Repo PR runs to fail with git exit code 128. This blocks validation of push-to-PR-branch functionality.
• Acceptance Criteria:
  • pr-branch exists in githubnext/gh-aw-side-repo with an open PR targeting main
  • Smoke Update Cross-Repo PR runs successfully (no push_to_pull_request_branch failure)
  • The branch is auto-recreated if deleted (self-healing smoke test)
• Technical Approach: Either (a) manually recreate the PR branch in the side-repo, or (b) add a smoke test setup step that creates/resets the branch before the agent runs
• Estimated Effort: Small

Work Item 2: Fix EP006 — Smoke Multi PR add_comment context handling (Medium)

• Type: Bug Fix (prompt + safe output handler)
• Priority: Medium
• Description: Smoke Multi PR's agent produces add_comment with target=triggering after creating a PR, but the workflow is triggered via workflow_dispatch which has no triggering issue/PR context. This causes an error every time.
• Acceptance Criteria:
  • No EP006 failures in Smoke Multi PR for 5 consecutive days
  • Agent correctly references the newly-created PR for comments (or uses noop if no context)
• Technical Approach: Update the Smoke Multi PR agent prompt to use item_number: created_pr_number (referencing the just-created PR) instead of target=triggering. Alternatively, add a handler-level grace for add_comment failures on dispatch runs.
• Estimated Effort: Small

---

Historical Context

7-Day Trend

Date	Runs	SO Failures	Success Rate	Notable
2026-03-16	239	4	81.8%	EP006 x2, EP015 x1, EP016 x1
2026-03-17	13	1	93.3%	GitHub App token 404
2026-03-18	176	1	83.3%	EP018 NEW: protected_path_prefixes block
2026-03-19	120	0	100.0%	Clean day
2026-03-20	98	4	90.2%	EP006 x2, EP015 x1, EP016 x1
2026-03-21	79	2	97.3%	EP016 x1, EP006 x1
2026-03-22	110	2	93.8%	EP006 x1, EP022 NEW x1

Trend: Safe output health is stable with 2-4 failures per day in smoke test workflows. Today's failures are both limited to smoke testing infrastructure — no production workflows affected. EP006 (add_comment context mismatch) is the most persistent recurring pattern (8+ occurrences across 6 days).

---

Metrics and KPIs

• Overall Safe Output Job Success Rate (today): 93.8% (excluding skipped)
• Most Reliable Job Type: All production workflow safe output jobs succeeded
• Most Problematic Job Types: push_to_pull_request_branch (EP022), add_comment context mode (EP006)
• Consecutive Days Without Production Failures: All failures today limited to smoke test infrastructure

Next Steps

• Recreate pr-branch in githubnext/gh-aw-side-repo (critical — blocks Smoke Update Cross-Repo PR)
• Update Smoke Multi PR agent prompt to use item_number instead of target=triggering
• Track EP022 for recurrence to determine if this is a one-off or systemic issue
• Consider adding a branch existence pre-flight check to the push_to_pull_request_branch handler

References:

• §23392658504 — Smoke Update Cross-Repo PR (push_to_pull_request_branch failure)
• §23392726144 — Smoke Multi PR (add_comment context failure)
• §23395585444 — This audit run

AI generated by Safe Output Health Monitor · history

• expires on Mar 23, 2026, 4:34 AM UTC

[pelikhan]

/plan

[github-actions[bot]]

🚀 Plan Command has started processing this discussion comment

[github-actions[bot]]

This discussion has been marked as outdated by Safe Output Health Monitor.

A newer discussion is available at Discussion #22376.