Skip to content

[aw-failures] P0/P1 Failure Report (2026-04-28 07:00–13:00 UTC): awf-api-proxy health check + copilot node-not-found #28947

@github-actions

Description

@github-actions

Executive Summary

Analysis of the last 6 hours found 28 failed/cancelled runs out of 40 total. Two distinct failure clusters require action. One false-positive issue was also found and closed.

P0 — awf-api-proxy container intermittently fails health check: At least 3 confirmed runs failed before agent activation across different engines (Sub-Issue Closer/copilot, Daily Team Evolution/claude, Smoke CI/copilot on PR trigger). The awf-api-proxy container starts but its health check times out, blocking docker compose up entirely.

P1 — node: command not found in copilot agent container: At least 2 copilot-engine scheduled workflows fail inside the container because node is not on PATH (Daily Issues Report Generator, Daily News).

27 PR-triggered workflow cancellations (faq/issue-484*, faq/issue-485* branches) are normal concurrency-cancel behavior — no action needed.

Failure Clusters

Cluster Severity Confirmed Runs Affected Workflows
awf-api-proxy unhealthy P0 ≥3 Smoke CI, Sub-Issue Closer, Daily Team Evolution
node: command not found P1 ≥2 Daily Issues Report Generator, Daily News, Copilot PR NLP Analysis
PR workflow cancellations Normal 27 Multiple (concurrency cancel)
False-positive issue Resolved 1 Daily Token Consumption Report (#28922 closed)

P0 Evidence: awf-api-proxy unhealthy

All three affected runs show the identical failure in agent-stdio.log:

Container awf-api-proxy  Waiting
Container awf-squid      Healthy
Container awf-api-proxy  Error
dependency failed to start: container awf-api-proxy is unhealthy
[ERROR] Failed to start containers: Error: Command failed with exit code 1: docker compose up -d --pull never

The failure is intermittent: runs 25048485748, 25048535353, and 25049687892 all started the api-proxy successfully on the same day and the same runner tier.

Affected run IDs and timing
Run ID Workflow Engine Time (UTC) Issue
§25049338576 Sub-Issue Closer copilot 11:10 #28917
§25049605437 Daily Team Evolution Insights claude 11:14 #28919
§25052667955 Smoke CI copilot 12:25 #28858 comment

Baseline (successful): §25048535353 — api-proxy started healthy same window.

P1 Evidence: node: command not found

Copilot-engine workflows that require Node.js tooling fail inside the agent container because node is not on PATH. Containers start successfully; failure occurs at agent invocation.

/bin/bash: line 1: node: command not found
Affected run IDs
Run ID Workflow Time (UTC) Issue
§25048535353 Daily Issues Report Generator 10:49 #28914
§25044675410 Daily News 09:25 #28885

Existing Issue Correlation

Proposed Fix Roadmap

Priority Fix Sub-Issue
P0 Fix awf-api-proxy health check intermittent failure #28949
P1 Ensure node is available on PATH in copilot agent container (existing #28914, #28885)

References:

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions