grammar: make MAX_REPETITION_THRESHOLD configurable via env var#21139
Open
vampyrebat wants to merge 1 commit intoggml-org:masterfrom
Open
grammar: make MAX_REPETITION_THRESHOLD configurable via env var#21139vampyrebat wants to merge 1 commit intoggml-org:masterfrom
vampyrebat wants to merge 1 commit intoggml-org:masterfrom
Conversation
The hardcoded threshold of 2000 causes grammar parsing to fail for legitimate tool-calling schemas with many optional parameters. Add LLAMA_GRAMMAR_MAX_REPETITIONS env var to override the default. When unset, behaviour is unchanged (default 2000). May fix grammar failures reported in openclaw/openclaw#32916, openclaw/openclaw#38569, openclaw/openclaw#38899.
Member
|
Duplicate of #21003 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The hardcoded threshold of 2000 causes grammar parsing to fail for legitimate tool-calling schemas with many optional parameters.
Add LLAMA_GRAMMAR_MAX_REPETITIONS env var to override the default. When unset, behaviour is unchanged (default 2000).
May fix grammar failures reported in openclaw/openclaw#32916, openclaw/openclaw#38569, openclaw/openclaw#38899.
AI was used in an assistive capacity to identify the threshold constant and review the approach. The code change and testing were done manually.
Overview
Adds a helper function that reads LLAMA_GRAMMAR_MAX_REPETITIONS from the environment, replacing the hardcoded MAX_REPETITION_THRESHOLD macro. Three call sites updated. No API changes, fully backwards compatible.
Additional information
Tested on with Qwen3.5-122B-A10B and 21 OpenClaw tools (message tool: 109 optional params, browser: 48). Default threshold (2000): grammar fails on every request. Threshold at 20000: grammar parses successfully.
Requirements