Implement a generic AIMD rate limit throttle

[westonpace]

Parent Issue

Part of OSS-636 (Implement AIMD Adaptive Retry)

Summary

Implement a generic AIMD (Additive Increase / Multiplicative Decrease) rate limit throttle as a standalone, reusable component. This should be a leaky bucket (or potentially GCRA) throttle whose rate is controlled by the AIMD algorithm.

Details

The throttle should support:

• Configurable parameters: initial rate, min rate, max rate, reduction factor, increase increment, adjust window, max attempts
• Rate adjustment logic:
  • Multiplicative Decrease: When throttled, reduce rate by the reduction factor (e.g., halve it), enforcing a minimum floor
  • Additive Increase: When successful, slowly increase rate by a fixed increment, enforcing an optional ceiling
• Failure categorization: Classify errors as throttling/congestion (triggers rate decrease), recoverable (retry only), or unrecoverable (fail immediately)
• Window-based decisions: Track request/throttle counts over a configurable window to decide when to adjust rates

The simplest knob to use may be the number of concurrent requests (concurrency), but a rate limit with an AIMD-controlled leaky bucket throttle is proposed. An AIMD-controlled concurrency limit would also be acceptable.

Open Questions

• Should rate limits be global across all requests, or should there be independent limits for read (including head), write, and list operations?