feat: forward sampling provider_opts (top_k, repetition_penalty) to provider APIs#2226
Merged
dgageot merged 2 commits intodocker:mainfrom Mar 23, 2026
Conversation
![docker-agent[bot]](https://avatars.githubusercontent.com/in/2787757?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
dgageot
force-pushed
the
board/docker-agent-issue-2220-feasibility-chec-c088f140
branch
from
a88e73a to
56a3913
Compare
…) to provider APIs Add support for passing sampling parameters via provider_opts to all provider backends. This enables custom OpenAI-compatible providers (vLLM, Ollama, llama.cpp) to receive parameters like top_k, repetition_penalty, min_p, and seed that they support but the native OpenAI API does not. Provider support: - OpenAI/custom: top_k, repetition_penalty, min_p, typical_p via SetExtraFields; seed via native field - Anthropic: top_k via native TopK field - Gemini: top_k via native TopK field - Bedrock: top_k via AdditionalModelRequestFields Also refactors Bedrock buildAdditionalModelRequestFields to avoid early returns that would discard top_k when thinking budget is invalid, and extracts isThinkingEnabled() to decouple thinking detection from additional fields presence. Assisted-By: docker-agent
…r provider_opts mismatches - Default interleaved_thinking to true, matching schema documentation - Warn when thinking_budget is set but interleaved_thinking is explicitly disabled - Add debug logging in providerutil when provider_opts values have wrong types Assisted-By: docker-agent
dgageot
force-pushed
the
board/docker-agent-issue-2220-feasibility-chec-c088f140
branch
from
56a3913 to
4520285
Compare
gtardif
approved these changes
Mar 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #2220
Adds support for passing sampling parameters via
provider_optsto all provider backends. This enables custom OpenAI-compatible providers (vLLM, Ollama, llama.cpp) to receive parameters liketop_k,repetition_penalty,min_p, andseed.Provider support
top_krepetition_penaltymin_pseedExample
Changes
pkg/model/provider/providerutilpackage with helpers for extracting typed values fromprovider_optsSetExtraFields;seedset nativelyTopKfield on both standard and beta clientsTopKonGenerateContentConfigtop_kviaAdditionalModelRequestFields; refactored thinking detection to avoid losingtop_kon early returnsagent-schema.jsonwith documentation for new sampling parameters