Skip to content

Feature: Circuit breaker for model rate limiting in Ralph#451

Merged
bradygaster merged 1 commit intobradygaster:devfrom
tamirdresher:feature/ralph-circuit-breaker
Mar 20, 2026
Merged

Feature: Circuit breaker for model rate limiting in Ralph#451
bradygaster merged 1 commit intobradygaster:devfrom
tamirdresher:feature/ralph-circuit-breaker

Conversation

@tamirdresher
Copy link
Copy Markdown
Collaborator

Summary

Adds a circuit breaker pattern for automatic model fallback when Copilot rate limits are hit during Ralph operations.

Closes #450

Problem

When running multiple Ralph instances, premium model rate limits cause cascading failures — all Ralphs go down at once.

Solution

Classic circuit breaker (Hystrix/Polly pattern) applied to model selection:

  • CLOSED (normal): Use preferred model (default: \claude-sonnet-4.6)
  • OPEN (rate limited): Fall back through free-tier chain: \gpt-5.4-mini\ → \gpt-5-mini\ → \gpt-4.1\
  • HALF-OPEN (testing): After cooldown, try preferred model. 2 consecutive successes → close circuit.

What's included

  • .squad-templates/ralph-circuit-breaker.md\ — Full documentation with:
    • Pattern explanation and state diagram
    • PowerShell functions (\Get-CurrentModel, \Update-CircuitBreakerOnSuccess, \Update-CircuitBreakerOnRateLimit)
    • State file schema (.squad/ralph-circuit-breaker.json)
    • Integration instructions for
      alph-watch.ps1\
    • Metrics tracking (fallbacks, recoveries)

Key design decisions

  • Free-tier models (0x multiplier) as fallbacks — unlimited usage, no quota burn
  • Cooldown-based recovery (default 10 min) — avoids hammering a rate-limited endpoint
  • Persistent state file — survives Ralph restarts
  • Cascading fallback chain — if even the first fallback is limited, tries the next

Testing

Validated with 8 concurrent Ralphs across 8 repos. Rate limit hit after ~2 hours of intensive polling. Circuit breaker engaged, Ralphs continued on \gpt-5.4-mini, recovered to preferred model after 10-minute cooldown.

@bradygaster bradygaster changed the base branch from main to dev March 20, 2026 03:54
@bradygaster bradygaster reopened this Mar 20, 2026
@bradygaster bradygaster reopened this Mar 20, 2026
@bradygaster bradygaster changed the base branch from dev to main March 20, 2026 04:21
@bradygaster bradygaster changed the base branch from main to dev March 20, 2026 04:21
Implements automatic model fallback when Copilot rate limits are hit:
- Preferred model -> free-tier fallback chain
- Cooldown-based recovery (half-open -> closed)
- Persistent state in .squad/ralph-circuit-breaker.json
- Tracks metrics: total fallbacks, recoveries

Follows classic circuit breaker pattern (Hystrix/Polly/Resilience4j)
applied to AI model selection.

Closes bradygaster#450

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bradygaster bradygaster force-pushed the feature/ralph-circuit-breaker branch from f466cca to 336a0a0 Compare March 20, 2026 08:23
@bradygaster bradygaster merged commit 04a44fa into bradygaster:dev Mar 20, 2026
2 checks passed
chrislomonico pushed a commit to clomonico/squad that referenced this pull request Mar 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: Circuit breaker for model rate limiting in Ralph

2 participants