Skip to content

Spec a CI-green gate with bounded fix-and-retry loop for iteration acceptance#42

Merged
mrjf merged 3 commits intomainfrom
copilot/fix-gate-iteration-acceptance
Apr 23, 2026
Merged

Spec a CI-green gate with bounded fix-and-retry loop for iteration acceptance#42
mrjf merged 3 commits intomainfrom
copilot/fix-gate-iteration-acceptance

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 22, 2026

Today an iteration is marked ✅ Accepted as soon as the sandbox-computed metric improves, but the agent sandbox often can't install project toolchains (tsc, cargo, pytest, …) so self-evaluation rubber-stamps broken commits onto the long-running branch with no revert or retry. Acceptance needs to be gated on real CI on the pushed HEAD, with fix-and-retry preferred over revert.

This PR is the spec change in workflows/autoloop.md only. The shared failure-signature extractor is deferred to sibling issue #34 (scheduler extraction).

Step 5 split into 5a / 5b / 5c

  • 5a — Push and wait for CI: commit, push, ensure draft PR exists, then gh pr checks --watch and reduce all check-runs to a single success / failure / pending via the awk pipeline from the issue.
  • 5b — Fix loop: on red, fetch logs, build a normalized failure signature, no-progress guard (pause if signature repeats), 5-attempt budget, 60-min wall-clock cap. Loops back to 5a after each fix push — never reverts.
  • 5c — Accept: only on green; existing accept actions plus fix-attempt count when > 0.
status=$(gh pr checks "$PR" --json conclusion,state -q '.[] | (.conclusion // .state // "")' \
  | awk '
      BEGIN { r = "success" }
      /^(FAILURE|CANCELLED|TIMED_OUT|ACTION_REQUIRED|STARTUP_FAILURE|STALE)$/ { r = "failure" }
      /^(PENDING|QUEUED|IN_PROGRESS|WAITING|REQUESTED)$/ { if (r == "success") r = "pending" }
      END { print r }')

pending is explicitly handled: never accepted, re-runs the watch step, falls through to ci-timeout if it persists past the wall-clock cap.

Machine State vocabulary

  • pause_reason: documents ci-fix-exhausted: <signature>, stuck in CI fix loop: <signature>, ci-timeout.
  • recent_statuses: adds ci-fix-exhausted as the coarse bucket for any CI-gate failure mode (fine-grained reason lives in pause_reason). Existing parser accepts arbitrary status words — no scheduler code change needed.
  • Iteration History entry format: optional CI fix attempts field.

Coordination

New subsection notes that PR-health-keeper / Evergreen workflows can pick up paused Autoloop PRs via the structured pause_reason field — same handoff as for human-authored PRs.

Preserved unchanged

The "metric did not improve" and "evaluation could not run" branches of Step 5 are untouched.

Copilot AI changed the title [WIP] Fix gate iteration acceptance on CI green with retry loop Spec a CI-green gate with bounded fix-and-retry loop for iteration acceptance Apr 22, 2026
Copilot finished work on behalf of mrjf April 22, 2026 22:32
Copilot AI requested a review from mrjf April 22, 2026 22:32
@mrjf
Copy link
Copy Markdown
Contributor

mrjf commented Apr 23, 2026

@copilot resolve the merge conflicts in this pull request

…ation-acceptance

# Conflicts:
#	workflows/autoloop.md

Co-authored-by: mrjf <180956+mrjf@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 23, 2026

@copilot resolve the merge conflicts in this pull request

Merged origin/main and resolved the conflicts in workflows/autoloop.md in 83b791f. Adopted main's "program issue" terminology (the steering-issue/experiment-log collapse from #41) while preserving all the CI-gate spec additions (5a/5b/5c, fix-attempt count, PR-health-keeper coordination subsection). All 109 tests still pass.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/repos/bogus.invalid/bogus/issues
    • Triggering command: /usr/bin/python /usr/bin/python /home/REDACTED/work/autoloop/autoloop/workflows/scripts/autoloop_scheduler.py (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot finished work on behalf of mrjf April 23, 2026 00:44
@mrjf mrjf marked this pull request as ready for review April 23, 2026 01:21
@mrjf mrjf merged commit 391371c into main Apr 23, 2026
1 check passed
@mrjf mrjf deleted the copilot/fix-gate-iteration-acceptance branch April 23, 2026 01:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Gate iteration acceptance on CI green, with a bounded fix-and-retry loop

2 participants