Skip to content

[CI Failure Doctor] Investigate update command integration test failure #18448

@github-actions

Description

@github-actions

CI Failure Investigation - Run #37900

Summary

Integration test TestUpdateCommand_NoMergeFlag in pkg/cli/update_integration_test.go now fails because gh aw update --no-merge exits successfully when there are no workflows with a source field, yet the test still asserts that the command should error out with "Should fail (no source workflows), output: ℹ no workflows found with source field." The CLI currently treats empty workflow sets as informational, so the integration test is out of sync with the implemented semantics.

Failure Details

  • Run: 22427845060
  • Commit: 645f972
  • Trigger: push
  • Job: Integration: CLI Update Command
  • Failure: pkg/cli/update_integration_test.go:170 TestUpdateCommand_NoMergeFlag expected a non-zero exit due to "no workflows found," but the command now exits zero after printing only the informational message.

Root Cause Analysis

UpdateWorkflows returns nil (with the informational message no workflows found with source field) whenever the scan finds zero workflows and no explicit workflow names are requested. TestRunUpdateWorkflows_NoSourceWorkflows (pkg/cli/update_command_test.go, ~line 971) already verifies that RunUpdateWorkflows should succeed in that situation, so the integration test is asserting the opposite behavior on the full binary. The CLI and unit test are aligned, but the integration test still assumes the older error-on-empty behavior.

Investigation Findings
  • Integration job log shows TestUpdateCommand_NoMergeFlag is failing because err is nil while the output contains "ℹ no workflows found with source field".
  • The unit test TestRunUpdateWorkflows_NoSourceWorkflows explicitly asserts that empty workflow directories should not be treated as an error.
  • No other tests failed in the job after this assertion.

Recommended Actions

  • Update TestUpdateCommand_NoMergeFlag (pkg/cli/update_integration_test.go) to expect success when no workflows have a source field and confirm --no-merge is parsed (e.g., keep the "unknown flag" assertion but expect err == nil).
  • Document the intended gh aw update semantics for empty workflow sets so integration and unit tests use the same contract.

Prevention Strategies

Synchronize integration tests with command-level semantics whenever CLI error handling changes; shared helpers or clear comments about when a command should exit 0 help keep tests aligned.

AI Team Self-Improvement

When you change CLI error handling (e.g., how UpdateWorkflows reports "no targets"), update any integration tests that make assertions about exit codes or error messages to prevent drift.

Historical Context
  • TestRunUpdateWorkflows_NoSourceWorkflows has already baked in the “informational on empty sets” behavior, so the CLI has intentionally been hardened to exit 0 in that scenario.
  • This failure is new for this run; no existing issue matched this mismatch.

🩺 Diagnosis provided by CI Failure Doctor

To install this agentic workflow, run

gh aw add githubnext/agentics/workflows/ci-doctor.md@ea350161ad5dcc9624cf510f134c6a9e39a6f94d
  • expires on Feb 27, 2026, 4:40 AM UTC

Metadata

Metadata

Assignees

No one assigned

    Labels

    cookieIssue Monster Loves Cookies!

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions