Skip to content

Support metric_direction: lower in program frontmatter#51

Merged
mrjf merged 2 commits intomainfrom
copilot/support-metric-direction-lower
Apr 23, 2026
Merged

Support metric_direction: lower in program frontmatter#51
mrjf merged 2 commits intomainfrom
copilot/support-metric-direction-lower

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 23, 2026

Autoloop hard-coded "higher is better" everywhere — the metric-improved check, the halting condition, and the iteration-history delta. Lower-is-better programs (error, latency, cost, fitness ratio) had to invert their metric in the evaluator, which made target-metric inversions unintuitive and iteration comments hard to read.

---
schedule: every 6h
metric_direction: lower   # defaults to "higher" if omitted
target-metric: 0.9        # program completes when best_metric ≤ 0.9
---

Scheduler (workflows/scripts/autoloop_scheduler.py + .github/ mirror)

  • parse_program_frontmatter parses metric_direction (case-insensitive; metric_direction: and metric-direction: both accepted; quoted values OK). Invalid values warn and fall back to higher. Return tuple grew from 3 → 5 elements.
  • select_program returns a new selected_metric_direction element; due items carry metric_direction; legacy entries default to higher.
  • /tmp/gh-aw/autoloop.json gains selected_metric_direction.

Agent prompt (workflows/autoloop.md)

  • Documents selected_metric_direction in the inputs section and adds a dedicated Metric Direction subsection.
  • Step 5a "metric improved" check is direction-aware (new > best for higher, new < best for lower).
  • Iteration History delta is documented as signed by direction+delta improves under higher, -delta improves under lower.
  • Step 5c.7 + Halting Condition section explicitly read selected_metric_direction (>= vs <= against target-metric).
  • Machine State table + field reference gain a Metric Direction row (defaults to higher when absent — no migration needed).

Tests (tests/test_scheduling.py)

  • New parse_frontmatter_full helper exposes direction.
  • Existing select_program unpackings updated for the 7-tuple return.
  • Added coverage for direction parsing (default, explicit, invalid, quoted, dashed alias, case-insensitive), direction-aware improvement and halting semantics, and select_program plumbing including the legacy default.

Backward compatibility

Programs without the field — and due/state entries without the key — continue to behave as higher. No existing program changes shape.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/repos/bogus.invalid/bogus/issues
    • Triggering command: /usr/bin/python /usr/bin/python /home/REDACTED/work/autoloop/autoloop/workflows/scripts/autoloop_scheduler.py (http block)
    • Triggering command: /usr/bin/python /usr/bin/python /home/REDACTED/work/autoloop/autoloop/workflows/scripts/autoloop_scheduler.py ic (http block)
    • Triggering command: /usr/bin/python /usr/bin/python /home/REDACTED/work/autoloop/autoloop/workflows/scripts/autoloop_scheduler.py rget-metric|Machine State|Metric Direction (http block)
  • https://api.github.com/repos/bogus.invalid/bogus/pulls
    • Triggering command: /usr/bin/python /usr/bin/python /home/REDACTED/work/autoloop/autoloop/workflows/scripts/autoloop_scheduler.py (http block)
    • Triggering command: /usr/bin/python /usr/bin/python /home/REDACTED/work/autoloop/autoloop/workflows/scripts/autoloop_scheduler.py ic (http block)
  • https://api.github.com/repos/githubnext/autoloop/issues
    • Triggering command: /usr/bin/python3 python3 /home/REDACTED/work/autoloop/autoloop/workflows/scripts/autoloop_scheduler.py --destination-port 53 -j ACCEPT (http block)
    • Triggering command: /usr/bin/python3 python3 /home/REDACTED/work/autoloop/autoloop/workflows/scripts/autoloop_scheduler.py (http block)
  • https://api.github.com/repos/githubnext/autoloop/pulls
    • Triggering command: /usr/bin/python3 python3 /home/REDACTED/work/autoloop/autoloop/workflows/scripts/autoloop_scheduler.py --destination-port 53 -j ACCEPT (http block)
    • Triggering command: /usr/bin/python3 python3 /home/REDACTED/work/autoloop/autoloop/workflows/scripts/autoloop_scheduler.py (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI changed the title [WIP] Add support for metric_direction: lower in program frontmatter Support metric_direction: lower in program frontmatter Apr 23, 2026
Copilot AI requested a review from mrjf April 23, 2026 14:58
Copilot finished work on behalf of mrjf April 23, 2026 14:58
@mrjf mrjf marked this pull request as ready for review April 23, 2026 21:00
@mrjf mrjf merged commit 15270d0 into main Apr 23, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support metric_direction: lower in program frontmatter (lower-is-better programs)

2 participants