Skip to content

Extract the scheduler from the inline heredoc into a committed script #34

@mrjf

Description

@mrjf

Summary

Move the ~470-line scheduler in .github/workflows/autoloop.md / workflows/autoloop.md out of its inline node - << 'JSEOF' block into a committed script (e.g., scripts/autoloop_scheduler.py or .js). Swap the run: step body to a one-liner that invokes the script.

Motivation

The inline heredoc is at 470+ lines today and continues to grow as scheduling policy evolves. GitHub Actions' compiled run: expressions have a size limit, and installations that add deterministic tie-breaking, richer autoloop.json fields, or an "existing PR" lookup hit it quickly.

Secondary benefits, all universal:

  • Unit-testable. Inline heredocs can't be exercised without spinning up Actions. A committed script can be unit-tested against representative autoloop.json fixtures (see tests/ in this repo for a natural home).
  • Editor tooling / linting. LSP, type-checking, formatters all work on a real file and nothing works inside a heredoc string.
  • Diffs are readable in PRs. Heredoc changes show as one giant replaced string in gh-aw's compiled lock file; file diffs are line-level.
  • Extension points. Adding existing_pr / head_branch detection (see sibling issue Enforce the single-PR-per-program invariant: preserve-branch-name, existing_pr/head_branch pre-step outputs, explicit 'no suffixes' prose #35) doubles the scheduler's size. Keeping it inline becomes untenable.

Current state in upstream

workflows/autoloop.md lines 106–575 are a single inline node - << 'JSEOF' … JSEOF block holding the entire scheduler.

Proposed change

The entire step becomes:

  - name: Check which programs are due
    env:
      GITHUB_TOKEN: ${{ github.token }}
      GITHUB_REPOSITORY: ${{ github.repository }}
      AUTOLOOP_PROGRAM: ${{ github.event.inputs.program }}
    run: |
      python3 .github/workflows/scripts/autoloop_scheduler.py

…and scripts/autoloop_scheduler.py ships alongside workflows/autoloop.md in this repo. The installer (install.md) copies both into the consuming repo.

Language choice: Python or Node both work. Python is slightly preferable because every GitHub-hosted runner has Python pre-installed with stdlib-only needs (urllib, json, re, pathlib), and because the Autoloop ecosystem has already chosen Python elsewhere.

Sibling work this enables

Several other improvements proposed in parallel issues add logic to the scheduler. Each is ~30–60 more lines. Doing them inline would push the heredoc ~700 lines, which will fail run: compilation for many users. Extraction is a prerequisite:

  • Deterministic tie-breaking among never-run programs (so the first discovered program doesn't permanently starve the others).
  • Emitting existing_pr and head_branch to autoloop.json.
  • Matching both the exact canonical branch and legacy framework-suffixed branches when resolving existing PRs.

Tests

Add fixture-based tests in tests/:

  • Given 2 programs with different last_run in state, pick the more-overdue one.
  • Given 2 never-run programs with different schedules, pick the one with the shorter schedule (deterministic tie-break).
  • Given a state file missing, treat as first-run.
  • Given paused: true, skip and report in skipped with reason.
  • Given completed: true, skip.
  • Given FORCED_PROGRAM=<name>, bypass scheduling and pick that name regardless of last_run.

Related

Without this extraction, the other improvements in this batch of issues cannot land without bumping against the inline-expression size limit.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions