From fdffe36b8bc44c17d1bf3799f27244c572dfb031 Mon Sep 17 00:00:00 2001 From: functionstackx <47992694+functionstackx@users.noreply.github.com> Date: Fri, 17 Apr 2026 20:59:21 -0400 Subject: [PATCH] Document mandatory --use-chat-template for MTP scripts MTP (EAGLE-style spec decoding) is trained on chat-formatted inputs; omitting --use-chat-template silently regresses acceptance rate and produces misleading benchmark numbers. Several recent MTP configs were missing the flag, so make the rule explicit in AGENTS.md. Co-Authored-By: Claude Opus 4.7 (1M context) --- AGENTS.md | 1 + 1 file changed, 1 insertion(+) diff --git a/AGENTS.md b/AGENTS.md index 4ea8f0e5f..50c9c9c14 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -163,6 +163,7 @@ When working with benchmark configurations, use these valid values: - Source shared utilities: `source benchmark_lib.sh` - Functions: `check_env_vars()`, `wait_for_server_ready()`, `run_benchmark_serving()`, `run_eval()`, `append_lm_eval_summary()` - Parameters passed via environment variables +- **MTP scripts MUST pass `--use-chat-template` to `run_benchmark_serving` — no exceptions.** EAGLE-style speculative decoding is trained against chat-formatted inputs, so benchmarking against raw prompts silently regresses acceptance rate and produces misleading numbers. This applies to every `*_mtp.sh` script regardless of model, precision, or runner. ### Git