common : merge qwen3-coder and nemotron nano 3 parsers

[aldehir]

Users are experiencing several issues with Qwen3-Coder-Next. Until #18675 is merged in, this PR serves as a stop-gap by replacing the existing Qwen3-Coder parsing with the Nemotron Nano 3 PEG parsing variant already present.

This PR also adds parallel tool calling and fixes JSON schema support.

fixes #19382
fixes #19430
fixes #19304
supersedes #19503 and #19753

[pwilkin]

Can you please add some tests on schema parameters, especially an array (such as a "todolist" tool)?

[pwilkin]

I'll try to test it today with OpenCode to see how it works.

[bfroemel]

Can we also add the fix for #19513 ?

@pwilkin, I believe it was just removing the trim_trailing_space() filters from reasoning (L30) and content (L34) here,?

https://github.com/aldehir/llama.cpp/blob/e811d2263fc10d1b1d9ed036708d6c9b6f798a7f/common/chat-peg-parser.cpp#L25-L36

[aldehir]

Can we also add the fix for #19513 ?

As I understand it, the model generates the following tokens:

llama.cpp > curl http://localhost:8080/tokenize -d '{"content": "call:\n\n<tool_call>", "parse_special": true, "with_pieces": true}'
{"tokens":[{"id":6659,"piece":"call"},{"id":1447,"piece":":\n\n"},{"id":151657,"piece":"<tool_call>"}]}

But when we parse it, we strip \n (the template always adds a \n before <tool_call>), and we feed it back as:

llama.cpp > curl http://localhost:8080/tokenize -d '{"content": "call:\n<tool_call>", "parse_special": true, "with_pieces": true}'
{"tokens":[{"id":6659,"piece":"call"},{"id":510,"piece":":\n"},{"id":151657,"piece":"<tool_call>"}]}

Which causes issues? This makes sense to me. It's a bit annoying because ultimately it depends on the model + template. In this case, we need to strip a single \n because it is added back in the chat template.

[pwilkin]

@aldehir Oh, this is a fun one :)

This causes issues because in the tokenizer, :\n\n is a token. A separate one from ":". And apparently that's a trick used to train the model toolcalling - if the previous token was the ":\n\n", the next token is more likely to be the tool call start.

When you pass this back as ":\n", this is no longer the same token. Therefore, the model at some point gets confused and stops doing tool calls if enough of those bad historical calls are done because it doesn't know what to do after a token it wasn't trained to call tools on.

[aldehir]

This causes issues because in the tokenizer, :\n\n is a token. A separate one from ":". And apparently that's a trick used to train the model toolcalling - if the previous token was the ":\n\n", the next token is more likely to be the tool call start.

Upon further inspection, I am struggling to see the issue.

The chat template will strip the message content and append a \n.

        {%- if message.content is defined and message.content is string and message.content | trim | length > 0 %}
            {{- '\n' + message.content | trim + '\n' }}
        {%- endif %}

https://huggingface.co/Qwen/Qwen3-Coder-Next/blob/main/chat_template.jinja

Then, it prepends a \n for tool calls

            {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }

Assuming the end of the content is :\n\n, even if we stripped trailing whitespace, the template will add back the \n\n. Due to the trimming in the template, it actually doesn't matter if we strip or not.

[bfroemel]

@aldehir Are you sure that the template adds the newline again?

I believe the tool call preamble (as it occurs with codex use/potentially related to the responses API) is rendered as a "normal" assistant message:

        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>\n' }}

Anyway, I can confirm that currently it's an issue with this PR, @pwilkin's fix (#18675 (comment)) lead to the apparently expected format and the issue was gone.

[bfroemel]

oh, remembering - I fell for the same "trap" :) #18675 (comment) (marked it as off-topic, contains the actually rendered prompt missing the required newlines)

[aldehir]

oh, remembering - I fell for the same "trap" :) #18675 (comment) (marked it as off-topic, contains the actually rendered prompt missing the required newlines)

I see. It's a bit weird that the preamble would be its own message. This means the client needs to perform another request to trigger a tool call, which doesn't quite align with the idea that retaining :\n\n yields uninterrupted sessions. Unless the Responses API is incorrectly formatting the input to the template.

[aldehir]

Sure enough, the Responses API is the culprit:
https://github.com/ggml-org/llama.cpp/blob/b908baf1825b1a89afef87b09e22c32af2ca6548/tools/server/server-common.cpp#L1217C1-L1241C50

Rather than generate a single message with content + tool_call, it generates two messages. I think this needs to be fixed instead, because feeding two separate messages will undoubtedly differ from how the model was trained.

[bfroemel]

Here just for reference a snippet from a responses API request as generated by codex (this example misses the 2 newlines at the end of tool preamble message), input array elements:

    {
      "type": "message",
      "role": "assistant",
      "content": [
        {
          "type": "output_text",
          "text": "Let me search for the main agent loop or turn handling:"
        }
      ]
    },
    {
      "type": "function_call",
      "name": "exec_command",
      "arguments": "{\"cmd\":\"rg --files /home/b/work/codex-new/codex-rs/core/src | xargs -I{} rg -l \\\"turn\\\\|loop\\\" {} 2>/dev/null | head -20\",\"justification\":\"Find files related to turn/loop\"}",
      "call_id": "fc_o1wk8CXzBhlun0u8esiYzJjZuz58M5Iz"
    },
    {
      "type": "function_call_output",
      "call_id": "fc_o1wk8CXzBhlun0u8esiYzJjZuz58M5Iz",
      "output": "Chunk ID: 7054eb\nWall time: 0.4461 seconds\nProcess exited with code 0\nOriginal token count: 0\nOutput:\n"
    },
    {
      "type": "function_call",
      "name": "exec_command",
      "arguments": "{\"cmd\":\"find /home/b/work/codex-new/codex-rs/core -type f -name \\\"*.rs\\\" | head -50\",\"justification\":\"List core module files\"}",
      "call_id": "fc_VP2ElL5yP9Y2knTm3gTh79VGOmn8s39e"
    },

Not seeing a case where content + tool call could be currently possible with https://github.com/openingnow/llama.cpp/blob/292f6908cdc6abb5c38581e34fa141973e5aba82/tools/server/server-common.cpp#L1072

I believe in the responses API there is no way to have type "message" contain also "function_call" elements. https://developers.openai.com/api/reference/resources/responses#(resource)%20responses%20%3E%20(model)%20response_input_item%20%3E%20(schema)

[aldehir]

Yes, I believe this is called a "leaky abstraction" 😊.

We should add a post step in the conversion that merges consecutive assistant messages. That seems like the most correction solution.

[pwilkin]

Aight, merging this one and we'll do a followup on the Responses problem.

[aldehir]

@bfroemel expect to see a PR for the Responses API here shortly. If you don't mind, I'd like to enlist your services to help test it out.

[bfroemel]

Hmm, so the model generates content output + tool call(s) - everything in a single generation; so single response. codex (as a responses API client) processes that response and builds the conversation in the input array for the next request with tool call responses. There it ends up as several elements (output_text message, tool calls, tool responses).

IMO that's fine and wouldn't immediately require to do post processing (trying to get multiple responses input elements into single chat completions elements), as long as it ends up as the expected prompt for the model (I looked at the rendered prompt - it's imo indistinguishable)? but ofc fine with any other solution that might be more clean/maintainable - especially if @pwilkin's fix could be problematic for other models.

/edit: adding a rendered prompt snippet (with the required newlines)

<|im_start|>assistant
Let me look at the parse_turn_item function:

<|im_end|>
<|im_start|>assistant
<tool_call>
<function=exec_command>
<parameter=cmd>
grep -n "fn parse_turn_item" /home/b/work/codex-new/codex-rs/core/src/*.rs
</parameter>
<parameter=justification>
Find parse_turn_item function
</parameter>
</function>
</tool_call><|im_end|>
<|im_start|>user
<tool_response>
Chunk ID: d74fd7
Wall time: 0.0510 seconds
Process exited with code 0
Original token count: 32
Output:
/home/b/work/codex-new/codex-rs/core/src/event_mapping.rs:96:pub fn parse_turn_item(item: &ResponseItem) -> Option<TurnItem> {

</tool_response><|im_end|>
<|im_start|>assistant
<tool_call>
<function=exec_command>
<parameter=cmd>
sed -n '96,200p' /home/b/work/codex-new/codex-rs/core/src/event_mapping.rs
</parameter>
<parameter=justification>
Read parse_turn_item function
</parameter>
</function>
</tool_call><|im_end|>

[aldehir]

This is what I expect:

<|im_start|>assistant
Let me look at the parse_turn_item function:

<tool_call>
<function=exec_command>
<parameter=cmd>

The additional <|im_end|>...<|im_start|> will likely cause the model to in-context learn and not generate a tool call in a single generation.