-
Notifications
You must be signed in to change notification settings - Fork 295
Description
CI Failure Investigation - Run #37900
Summary
Integration test TestUpdateCommand_NoMergeFlag in pkg/cli/update_integration_test.go now fails because gh aw update --no-merge exits successfully when there are no workflows with a source field, yet the test still asserts that the command should error out with "Should fail (no source workflows), output: ℹ no workflows found with source field." The CLI currently treats empty workflow sets as informational, so the integration test is out of sync with the implemented semantics.
Failure Details
- Run: 22427845060
- Commit: 645f972
- Trigger: push
- Job: Integration: CLI Update Command
- Failure:
pkg/cli/update_integration_test.go:170TestUpdateCommand_NoMergeFlagexpected a non-zero exit due to "no workflows found," but the command now exits zero after printing only the informational message.
Root Cause Analysis
UpdateWorkflows returns nil (with the informational message no workflows found with source field) whenever the scan finds zero workflows and no explicit workflow names are requested. TestRunUpdateWorkflows_NoSourceWorkflows (pkg/cli/update_command_test.go, ~line 971) already verifies that RunUpdateWorkflows should succeed in that situation, so the integration test is asserting the opposite behavior on the full binary. The CLI and unit test are aligned, but the integration test still assumes the older error-on-empty behavior.
Investigation Findings
- Integration job log shows
TestUpdateCommand_NoMergeFlagis failing becauseerrisnilwhile the output contains "ℹ no workflows found with source field". - The unit test
TestRunUpdateWorkflows_NoSourceWorkflowsexplicitly asserts that empty workflow directories should not be treated as an error. - No other tests failed in the job after this assertion.
Recommended Actions
- Update
TestUpdateCommand_NoMergeFlag(pkg/cli/update_integration_test.go) to expect success when no workflows have asourcefield and confirm--no-mergeis parsed (e.g., keep the "unknown flag" assertion but expecterr == nil). - Document the intended
gh aw updatesemantics for empty workflow sets so integration and unit tests use the same contract.
Prevention Strategies
Synchronize integration tests with command-level semantics whenever CLI error handling changes; shared helpers or clear comments about when a command should exit 0 help keep tests aligned.
AI Team Self-Improvement
When you change CLI error handling (e.g., how UpdateWorkflows reports "no targets"), update any integration tests that make assertions about exit codes or error messages to prevent drift.
Historical Context
TestRunUpdateWorkflows_NoSourceWorkflowshas already baked in the “informational on empty sets” behavior, so the CLI has intentionally been hardened to exit 0 in that scenario.- This failure is new for this run; no existing issue matched this mismatch.
🩺 Diagnosis provided by CI Failure Doctor
To install this agentic workflow, run
gh aw add githubnext/agentics/workflows/ci-doctor.md@ea350161ad5dcc9624cf510f134c6a9e39a6f94d
- expires on Feb 27, 2026, 4:40 AM UTC