Skip to content

Gate autoloop iteration acceptance on CI green with a bounded fix-retry loop#176

Merged
mrjf merged 3 commits intomainfrom
copilot/fix-autoloop-iteration-acceptance
Apr 21, 2026
Merged

Gate autoloop iteration acceptance on CI green with a bounded fix-retry loop#176
mrjf merged 3 commits intomainfrom
copilot/fix-autoloop-iteration-acceptance

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 21, 2026

Today an autoloop iteration is marked ✅ Accepted on the strength of the in-sandbox self-evaluation, which is unreliable by construction (the sandbox can't always install bun/tsc due to firewall blocks on releaseassets.githubusercontent.com). CI runs after the push and is not in the acceptance path, so broken commits (e.g. #174's 24 TS2339 errors) stack up on the long-running branch unnoticed. This change moves the acceptance gate to CI and adds a bounded fix-and-retry loop — no reverts.

.github/workflows/autoloop.md

  • Step 5 split into 5 / 5a / 5b / 5c. Sandbox metric is now a cheap pre-filter only; acceptance requires CI green on the pushed HEAD.
  • 5a — push and wait for CI via gh pr checks --watch --interval 30 with a REST fallback. Hardened status parser handles pending / in_progress / neutral / skipped / cancelled / timed_out / action_required. 60-min per-iteration wall-clock cap.
  • 5b — fix-retry loop, max 5 attempts. Fetches --log-failed, extracts a structured summary + normalized failure signature, implements a minimum surgical fix, pushes, re-waits. No-progress guard: two consecutive identical signatures → stop. On exhaustion: paused: true with one of ci-fix-exhausted: <sig>, stuck in CI fix loop: <sig>, or ci-timeout: wall-clock cap reached, plus a program-issue comment listing the last three attempts. Never revert — partial progress is preserved for a human or Evergreen to continue from.
  • 5c — accept only on CI green. Clears pause_reason, records fix-attempt count and CI run link in iteration history / per-iteration comment.
  • Added ci-fix-exhausted to the Recent Statuses vocabulary; documented all recognized Pause Reason values in the Machine State field reference.
  • Added common-mistake entries: do not revert on red CI, do not mark accepted while CI is red.

.github/workflows/evergreen.md

  • New Autoloop PRs section. autoloop/* PRs already flow through the existing PR-health pipeline; on successful fix, Evergreen now also un-pauses the program: checks out memory/autoloop, sets Paused: false / Pause Reason: — in {program-name}.md, commits, and comments on the program issue. Same 5-attempts-per-HEAD-SHA cap still applies.

Example status command (Step 5a)

PR=${EXISTING_PR:-$(gh pr list --head autoloop/{program-name} --json number -q '.[0].number')}
gh pr checks "$PR" --watch --interval 30 || true
status=$(gh pr checks "$PR" --json conclusion,state -q '.[] | (.conclusion // .state // "")' \
  | awk '
      BEGIN { r = "success" }
      /^(FAILURE|CANCELLED|TIMED_OUT|ACTION_REQUIRED|STARTUP_FAILURE|STALE)$/ { r = "failure" }
      /^(PENDING|QUEUED|IN_PROGRESS|WAITING|REQUESTED)$/ { if (r == "success") r = "pending" }
      END { print r }')

No autoloop.lock.yml regeneration is required — it pulls the prompt at runtime via {{#runtime-import .github/workflows/autoloop.md}}.

Copilot AI changed the title [WIP] Fix autoloop to require green CI before accepting iterations Gate autoloop iteration acceptance on CI green with a bounded fix-retry loop Apr 21, 2026
Copilot finished work on behalf of mrjf April 21, 2026 21:58
Copilot AI requested a review from mrjf April 21, 2026 21:58
@mrjf mrjf marked this pull request as ready for review April 21, 2026 22:24
@mrjf mrjf merged commit 585b95e into main Apr 21, 2026
17 checks passed
@mrjf mrjf deleted the copilot/fix-autoloop-iteration-acceptance branch April 21, 2026 22:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Autoloop: don't accept an iteration until CI is green — fix-and-retry until it passes

2 participants