Skip to content

Commit faab1fc

Browse files
committed
doc: document the overrides option
1 parent b7bc700 commit faab1fc

File tree

1 file changed

+9
-0
lines changed

1 file changed

+9
-0
lines changed

docs/content/docs/advanced/advanced-usage.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -233,6 +233,15 @@ n_draft: 0
233233
# Quantization settings for the model, impacting memory and processing speed.
234234
quantization: ""
235235
236+
# List of KV Overrides for llama.cpp (--override-kv flag)
237+
# Format: KEY=TYPE:VALUE
238+
# Example: `qwen3moe.expert_used_count=int:10`
239+
# Use this to override model configuration values at runtime.
240+
# Supported types include: int, float, string, bool.
241+
# Multiple overrides can be specified as a list.
242+
overrides:
243+
- KEY=TYPE:VALUE
244+
236245
# Utilization percentage of GPU memory to allocate for the model. (vLLM)
237246
gpu_memory_utilization: 0
238247

0 commit comments

Comments
 (0)