Update the cache settings in Py SDK

[Spatika-Ganesh]

ref: WOR-2424: Cache Settings

[linear]

WOR-2424 Cache Settings

guillaume I suggest we change our cache settings to: — inspired by Braintrust.

The cache follows the following rules:

There are three caching modes: auto (default), always, never.
In auto mode, completions are cached if they have temperature=0.
In always mode, completions are cached.
In never mode, the cache is never read or written to.

currently, our default WHEN_AVAILABLE will cache completions even when the temperature is not 0 — which is a bit confusing I think. We even had to adjust our own playground logic to account…

What do you think?

For a migration, we can make WHEN_AVAILABLE the same than AUTO

---

• API changes
• SDK changes
• Code Generation changes