Skip agent failure issues when PR checkout fails #13397

Copilot · 2026-02-03T02:53:11Z

When a PR is merged and its branch deleted, the "Checkout PR branch" step fails, triggering spurious failure issues. The conclusion job now detects this scenario via job outputs and skips issue creation.

Changes

actions/setup/js/checkout_pr_branch.cjs

Added core.setOutput("checkout_pr_success", "true"/"false") to indicate checkout result
Sets output to "true" on success, "false" on failure, "true" when no PR context

actions/setup/js/handle_agent_failure.cjs

Reads GH_AW_CHECKOUT_PR_SUCCESS environment variable from agent job output
Skips issue creation when value is "false" (checkout failed)

Go Compiler Changes

pkg/workflow/pr.go: Added id: checkout-pr to the checkout step
pkg/workflow/compiler_activation_jobs.go: Added checkout_pr_success to agent job outputs
pkg/workflow/notify_comment.go: Passes checkout status to conclusion job via environment variable

Tests

Added 4 test cases in checkout_pr_branch.test.cjs for output behavior
Added 3 test cases in handle_agent_failure.test.cjs for failure detection via job outputs

Architecture

The solution uses native GitHub Actions job outputs to communicate checkout status from the agent job to the conclusion job, eliminating the need for Actions API calls.

Agent Job → outputs.checkout_pr_success → Conclusion Job (env var) → handle_agent_failure.cjs

Behavior

Checkout fails (PR merged): No issue created ✓
Agent fails (other reason): Issue created as before ✓
No PR checkout step: Defaults to "true", normal behavior ✓
No Actions API calls: Uses native job outputs mechanism ✓

Original prompt

When the checkout PR step fails, the conclusion job should ignore the agent failure as it typically means the PR got merged and the branch deleted. No agent failure issue/comment should be generated.

See this run: #13395

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

pelikhan

@copilot do not use actions APIs. Instead of actions and job outputs to carry a successful checkout PR value.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot · 2026-02-03T03:14:49Z

@copilot do not use actions APIs. Instead of actions and job outputs to carry a successful checkout PR value.

Done! Changed the implementation to use job outputs instead of the Actions API.

Changes made (commit b658822):

checkout_pr_branch.cjs - Sets checkout_pr_success output ("true"/"false")
Agent job - Exposes checkout_pr_success as a job output with || 'true' fallback
Conclusion job - Receives output via GH_AW_CHECKOUT_PR_SUCCESS environment variable
handle_agent_failure.cjs - Checks env var, skips issue creation when "false"

The solution now uses native GitHub Actions job outputs instead of API calls, as requested.

github-actions · 2026-02-03T03:19:48Z

📰 BREAKING: Smoke Copilot is now investigating this pull request. Sources say the story is developing...

github-actions · 2026-02-03T03:19:48Z

🎉 Yo ho ho! Changeset Generator found the treasure and completed successfully! ⚓💰

github-actions · 2026-02-03T03:19:49Z

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

github-actions · 2026-02-03T03:19:50Z

🎬 THE END — Smoke Claude MISSION: ACCOMPLISHED! The hero saves the day! ✨

github-actions · 2026-02-03T03:22:25Z

Agent Container Tool Check

Tool	Status	Version
bash	✅	`5.2.21(1)-release`
sh	✅	available
git	✅	`2.52.0`
jq	✅	`jq-1.7`
yq	✅	`v4.50.1`
curl	✅	`8.x`
gh	✅	`2.86.0`
node	✅	`v24.13.0`
python3	✅	`3.x`
go	✅	`go1.24.12`
java	❌	not found
dotnet	❌	not found

Result: 10/12 tools available ❌ FAIL

Missing Tools:

java - Java runtime not installed
dotnet - .NET runtime not installed

AI generated by Agent Container Smoke Test

github-actions · 2026-02-03T03:23:44Z

PR titles: Remove .vscode/tasks.json; Add noop tool call to changeset generator for skipped PRs
Tests: GitHub MCP ✅; safeinputs-gh ✅; Serena activate ✅; Playwright ❌ (tool unavailable); Tavily ✅; file write ✅; bash cat ✅; discussion comment ✅; build ✅
Overall status: FAIL

AI generated by Smoke Codex

github-actions · 2026-02-03T03:24:05Z

🤖 Beep boop! The smoke test agent was here testing discussion interactions! All systems nominal! 🚀✨

Automated testing in progress for run §21615547949

AI generated by Smoke Copilot

github-actions · 2026-02-03T03:24:06Z

Smoke Test Results

✅ GitHub MCP: Remove .vscode/tasks.json #13396, Add noop tool call to changeset generator for skipped PRs #13393
✅ Safe Inputs GH CLI: Propagate create-discussion errors to conclusion job failure tracking #13399, Skip agent failure issues when PR checkout fails #13397
✅ Serena MCP, File Writing, Bash, Discussion, Build
❌ Playwright: network restrictions

Overall Status: FAIL (8/9)

@pelikhan - Run: https://github.com/github/gh-aw/actions/runs/21615547949

AI generated by Smoke Copilot

github-actions · 2026-02-03T03:24:22Z

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

github-actions · 2026-02-03T03:24:47Z

Smoke Test Results

Last 2 Merged PRs:

Remove .vscode/tasks.json #13396: Remove .vscode/tasks.json
Add noop tool call to changeset generator for skipped PRs #13393: Add noop tool call to changeset generator for skipped PRs

Test Results:

✅ GitHub MCP
✅ Safe Inputs GH CLI
✅ Serena MCP
✅ Make Build
⚠️ Playwright (browser install failed)
✅ Tavily Search
✅ File Writing
✅ Bash Tool
✅ Discussion Interaction

Overall Status: PASS

AI generated by Smoke Claude

Copilot

Pull request overview

This PR fixes spurious agent failure issues that are created when a PR is merged and its branch is deleted, causing the "Checkout PR branch" step to fail. The solution uses GitHub Actions job outputs to communicate the checkout status from the agent job to the conclusion job, which then skips issue creation when the failure is only due to PR checkout.

Changes:

Added checkout_pr_success output to track PR checkout status, defaulting to "true" when the step is skipped or succeeds, and "false" when it fails
Modified failure handling logic to skip issue creation when checkout fails
Added comprehensive test coverage for the new behavior

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
`pkg/workflow/pr.go`	Added step ID `checkout-pr` to enable output reference
`pkg/workflow/compiler_activation_jobs.go`	Added `checkout_pr_success` job output with fallback to 'true'
`pkg/workflow/notify_comment.go`	Passes checkout status to conclusion job via `GH_AW_CHECKOUT_PR_SUCCESS` environment variable
`actions/setup/js/checkout_pr_branch.cjs`	Sets output to "true" on success/skip, "false" on failure
`actions/setup/js/handle_agent_failure.cjs`	Skips issue creation when agent fails due to PR checkout
`actions/setup/js/checkout_pr_branch.test.cjs`	Tests verify output is set correctly in all scenarios
`actions/setup/js/handle_agent_failure.test.cjs`	Tests verify issue creation is skipped when checkout fails
`.github/workflows/security-guard.lock.yml`	Compiled workflow reflects all changes

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-03T05:07:16Z

actions/setup/js/handle_agent_failure.cjs

+    // If checkout_pr_success is "false", skip creating an issue as this is expected behavior
+    if (agentConclusion === "failure" && checkoutPRSuccess === "false") {
+      core.info("Skipping failure handling - failure was due to PR checkout (likely PR merged)");


The condition should also check that there are no assignment errors or missing safe outputs before skipping issue creation. If a PR checkout fails but there are also assignment errors or missing safe outputs, an issue should still be created to report those problems. The condition should be: if (agentConclusion === "failure" && checkoutPRSuccess === "false" && !hasAssignmentErrors && !hasMissingSafeOutputs)

Suggested change

// If checkout_pr_success is "false", skip creating an issue as this is expected behavior

if (agentConclusion === "failure" && checkoutPRSuccess === "false") {

core.info("Skipping failure handling - failure was due to PR checkout (likely PR merged)");

// If checkout_pr_success is "false" and there are no assignment errors or missing safe outputs,

// skip creating an issue as this is expected behavior

if (agentConclusion === "failure" && checkoutPRSuccess === "false" && !hasAssignmentErrors && !hasMissingSafeOutputs) {

core.info("Skipping failure handling - failure was due to PR checkout (likely PR merged) and no additional issues were detected");

Copilot · 2026-02-03T05:07:16Z

actions/setup/js/handle_agent_failure.test.cjs

+  describe("checkout PR failure via output", () => {
+    it("should skip issue creation when checkout_pr_success is false", async () => {
+      // Set the checkout PR failure environment variable
+      process.env.GH_AW_CHECKOUT_PR_SUCCESS = "false";
+
+      await main();
+
+      // Verify that no issue was created
+      expect(mockGithub.rest.issues.create).not.toHaveBeenCalled();
+      expect(mockGithub.rest.issues.createComment).not.toHaveBeenCalled();
+      expect(mockCore.info).toHaveBeenCalledWith(expect.stringContaining("Skipping failure handling - failure was due to PR checkout"));
+    });
+
+    it("should create issue when checkout_pr_success is true", async () => {
+      // Set the checkout PR success environment variable
+      process.env.GH_AW_CHECKOUT_PR_SUCCESS = "true";
+
+      mockGithub.rest.search.issuesAndPullRequests
+        .mockResolvedValueOnce({
+          // First search: PR search (no PR found)
+          data: { total_count: 0, items: [] },
+        })
+        .mockResolvedValueOnce({
+          // Second search: parent issue
+          data: { total_count: 0, items: [] },
+        })
+        .mockResolvedValueOnce({
+          // Third search: failure issue
+          data: { total_count: 0, items: [] },
+        });
+
+      mockGithub.rest.issues.create
+        .mockResolvedValueOnce({
+          data: { number: 1, html_url: "https://example.com/1", node_id: "I_1" },
+        })
+        .mockResolvedValueOnce({
+          data: { number: 2, html_url: "https://example.com/2", node_id: "I_2" },
+        });
+
+      mockGithub.graphql = vi.fn().mockResolvedValue({
+        addSubIssue: {
+          issue: { id: "I_1", number: 1 },
+          subIssue: { id: "I_2", number: 2 },
+        },
+      });
+
+      await main();
+
+      // Verify issue was created
+      expect(mockGithub.rest.issues.create).toHaveBeenCalled();
+    });
+
+    it("should create issue when checkout_pr_success is not set", async () => {
+      // Don't set GH_AW_CHECKOUT_PR_SUCCESS (workflow without PR checkout)
+      delete process.env.GH_AW_CHECKOUT_PR_SUCCESS;
+
+      mockGithub.rest.search.issuesAndPullRequests
+        .mockResolvedValueOnce({
+          // First search: PR search (no PR found)
+          data: { total_count: 0, items: [] },
+        })
+        .mockResolvedValueOnce({
+          // Second search: parent issue
+          data: { total_count: 0, items: [] },
+        })
+        .mockResolvedValueOnce({
+          // Third search: failure issue
+          data: { total_count: 0, items: [] },
+        });
+
+      mockGithub.rest.issues.create
+        .mockResolvedValueOnce({
+          data: { number: 1, html_url: "https://example.com/1", node_id: "I_1" },
+        })
+        .mockResolvedValueOnce({
+          data: { number: 2, html_url: "https://example.com/2", node_id: "I_2" },
+        });
+
+      mockGithub.graphql = vi.fn().mockResolvedValue({
+        addSubIssue: {
+          issue: { id: "I_1", number: 1 },
+          subIssue: { id: "I_2", number: 2 },
+        },
+      });
+
+      await main();
+
+      // Verify issue was created (normal failure handling)
+      expect(mockGithub.rest.issues.create).toHaveBeenCalled();
+    });
+  });


Missing test coverage for the edge case where both checkout fails (checkout_pr_success is false) AND there are assignment errors or missing safe outputs. In this scenario, an issue should still be created to report the assignment errors/missing outputs, but there's no test verifying this behavior.

Initial plan

c9b1a2b

Copilot AI assigned Copilot and pelikhan Feb 3, 2026

Copilot started work on behalf of pelikhan February 3, 2026 02:53 View session

Add checkout PR failure detection to skip issue creation

c681cd2

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Update conclusion job to ignore agent failure on PR merge~~ Skip agent failure issues when PR checkout fails Feb 3, 2026

Copilot AI requested a review from pelikhan February 3, 2026 03:02

Copilot finished work on behalf of pelikhan February 3, 2026 03:02

pelikhan reviewed Feb 3, 2026

View reviewed changes

Copilot started work on behalf of pelikhan February 3, 2026 03:05 View session

github-actions bot mentioned this pull request Feb 3, 2026

Smoke Test: Copilot - 21615204539 #13398

Closed

Use job outputs instead of Actions API for checkout failure detection

b658822

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI requested a review from pelikhan February 3, 2026 03:15

Copilot finished work on behalf of pelikhan February 3, 2026 03:15

pelikhan added the smoke label Feb 3, 2026

github-actions bot mentioned this pull request Feb 3, 2026

Smoke Test: Copilot - 21615547949 #13400

Closed

github-actions bot mentioned this pull request Feb 3, 2026

Smoke Test: Claude - 21615547953 #13401

Closed

github-actions bot added the smoke-claude label Feb 3, 2026

pelikhan marked this pull request as ready for review February 3, 2026 05:02

Copilot AI review requested due to automatic review settings February 3, 2026 05:02

pelikhan merged commit e483829 into main Feb 3, 2026
47 checks passed

pelikhan deleted the copilot/ignore-agent-failure-on-pr-merge branch February 3, 2026 05:02

Copilot started reviewing on behalf of pelikhan February 3, 2026 05:02 View session

Copilot AI reviewed Feb 3, 2026

View reviewed changes

github-actions bot mentioned this pull request Feb 3, 2026

[team-status] 🌟 Daily Team Status - February 3, 2026 #13442

Open

-    // If checkout_pr_success is "false", skip creating an issue as this is expected behavior
-    if (agentConclusion === "failure" && checkoutPRSuccess === "false") {
-      core.info("Skipping failure handling - failure was due to PR checkout (likely PR merged)");
+    // If checkout_pr_success is "false" and there are no assignment errors or missing safe outputs,
+    // skip creating an issue as this is expected behavior
+    if (agentConclusion === "failure" && checkoutPRSuccess === "false" && !hasAssignmentErrors && !hasMissingSafeOutputs) {
+      core.info("Skipping failure handling - failure was due to PR checkout (likely PR merged) and no additional issues were detected");

Skip agent failure issues when PR checkout fails #13397

Skip agent failure issues when PR checkout fails #13397

Uh oh!

Conversation

Copilot AI commented Feb 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Architecture

Behavior

Uh oh!

pelikhan left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI commented Feb 3, 2026

Uh oh!

github-actions bot commented Feb 3, 2026

Uh oh!

github-actions bot commented Feb 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 3, 2026

Agent Container Tool Check

Uh oh!

github-actions bot commented Feb 3, 2026

Uh oh!

github-actions bot commented Feb 3, 2026

Uh oh!

github-actions bot commented Feb 3, 2026

Smoke Test Results

Uh oh!

github-actions bot commented Feb 3, 2026

Uh oh!

github-actions bot commented Feb 3, 2026

Smoke Test Results

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Copilot AI commented Feb 3, 2026 •

edited

Loading

github-actions bot commented Feb 3, 2026 •

edited

Loading

github-actions bot commented Feb 3, 2026 •

edited

Loading

github-actions bot commented Feb 3, 2026 •

edited

Loading