Eval bug: llama3.2 answering a single JSON line

[kpouget]

Name and Version

llama-cli, llama-server b7849

Operating systems

Mac

GGML backends

Metal

Hardware

M4 Pro

Models

llama3.2

Problem description & steps to reproduce

With the recent versions of llama.cpp, I hit this template issue:

> tell what is GGML API

{"name": "tell", "parameters": {"tt": "GGML API"}}

[ Prompt: 832.0 t/s | Generation: 82.2 t/s ]

First Bad Commit

Git bisection pointed to the merge of this PR: #18462

Relevant log output

Other model families don't hit this issue

ramalama   run  smollm:135m
🦭 > hello
Hello! How can I help you?

ramalama   run  ibm/granite:2b
🦭 > hello
Hello! It's a pleasure to meet you. How can I assist you today?

[ggerganov]

@CISC @ngxson @pwilkin Could you take a look. I can reproduce with this model:

llama-cli -hf bartowski/Llama-3.2-3B-Instruct-GGUF --verbose

ref: https://huggingface.co/bartowski/Llama-3.2-3B-Instruct-GGUF

The issue seems like the chat template is not parsed properly and even though there should be no tools defined, it still injects the tools-related system prompt, which causes the model to try to use tools:

0.02.651.232 I init: chat template, example_format: '<|start_header_id|>system<|end_header_id|>

Environment: ipython
Cutting Knowledge Date: December 2023
Today Date: 28 Jan 2026

You have access to the following functions. To call a function, please respond with JSON for a function call.Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.Do not use variables.

You are a helpful assistant<|eot_id|><|start_header_id|>user<|end_header_id|>

Hello<|eot_id|><|start_header_id|>assistant<|end_header_id|>

Hi there<|eot_id|><|start_header_id|>user<|end_header_id|>

How are you?<|eot_id|><|start_header_id|>assistant<|end_header_id|>

'
0.02.651.431 I srv          init: init: chat template, thinking = 0

[CISC]

The issue seems like the chat template is not parsed properly and even though there should be no tools defined, it still injects the tools-related system prompt, which causes the model to try to use tools:

This is because the template does this:

{%- if not tools_in_user_message is defined %}
    {%- set tools_in_user_message = true %}
{%- endif %}
{%- if not tools is defined %}
    {%- set tools = none %}
{%- endif %}

then later checks with

{%- if tools_in_user_message and not tools is none %}

..and AFAICR we set tools = [] to avoid issues with other templates that didn't like tools being undefined, however I think #19147 is about to fix that.

[ggerganov]

Ok thanks. Btw, I did try #19147 before posting, but it still behaved like this.

[CISC]

Ok thanks. Btw, I did try #19147 before posting, but it still behaved like this.

Yeah, it will need an additional fix (not setting tools = []).