Skip to content

jinja : add missing 'in' test to template engine (#19004)#19239

Merged
ngxson merged 2 commits intoggml-org:masterfrom
sidmohan0:fix/jinja-in-test
Feb 2, 2026
Merged

jinja : add missing 'in' test to template engine (#19004)#19239
ngxson merged 2 commits intoggml-org:masterfrom
sidmohan0:fix/jinja-in-test

Conversation

@sidmohan0
Copy link
Copy Markdown
Contributor

@sidmohan0 sidmohan0 commented Feb 1, 2026 •
edited
Loading

Hey all!

Note: Fixes #19004

Context:

The jinja template parser was missing the 'in' test from global_builtins(), causing templates using reject("in", ...), select("in", ...), or 'x is in(y)' to fail with selectattr: unknown test 'in'

This broke tool-calling for Qwen3-Coder and would break for any other model whose chat template uses the in test.

What I did:

  • Added test_is_in supporting array, string, and object containment checks, mirroring the existing 'in' operator logic in runtime.cpp.
  • Includes test cases for all three containment types plus reject/select filter usage.

Local Test environment:

  • Windows 11 26200 (x64), Intel i7-10700K, 64GB RAM
  • MSVC 17.14.25 (VS Build Tools 2022)
  • CMake 4.2.0-rc4
  • All 279 jinja tests pass (1241 assertions, 0 failures, 0 exceptions)

Comments:

  • Shouldn't impact existing entries as we're just adding a new entry; the only way this code runs is if a template explicitly uses the in test

AI Disclosure

Built with Claude (AI-assisted). Fully tested locally. I understand what the code does.

Appreciate the look - let me know if you have any questions!

@sidmohan0 @claude
jinja : add missing 'in' test to template engine (ggml-org#19004)
421ffab 
The jinja template parser was missing the 'in' test from
global_builtins(), causing templates using reject("in", ...),
select("in", ...), or 'x is in(y)' to fail with
"selectattr: unknown test 'in'".

This broke tool-calling for Qwen3-Coder and any other model
whose chat template uses the 'in' test.

Added test_is_in supporting array, string, and object containment
checks, mirroring the existing 'in' operator logic in runtime.cpp.

Includes test cases for all three containment types plus
reject/select filter usage.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@github-actions github-actions Bot added testing Everything test related jinja parser Issues related to the jinja parser labels Feb 1, 2026
@loci-dev loci-dev mentioned this pull request Feb 1, 2026
Open
@ngxson
Copy link
Copy Markdown
Contributor

ngxson commented Feb 1, 2026

This PR appears to contain AI-generated code without proper disclosure or watermarking. Per our contribution guidelines, we cannot accept it in its current form.

@sidmohan0
Copy link
Copy Markdown
Contributor Author

sidmohan0 commented Feb 1, 2026

This PR appears to contain AI-generated code without proper disclosure or watermarking. Per our contribution guidelines, we cannot accept it in its current form.

Thank you for flagging - just updated the PR description. Sincere apologies for the oversight - genuinely did not mean to mislead.

@ngxson ngxson merged commit 0dfcd3b into ggml-org:master Feb 2, 2026
69 of 78 checks passed
shaofeiqi pushed a commit to qualcomm/llama.cpp that referenced this pull request Feb 6, 2026
@sidmohan0 @claude
@ngxson @shaofeiqi
jinja : add missing 'in' test to template engine (ggml-org#19004) (gg…
b9e8a5d 
…ml-org#19239)

* jinja : add missing 'in' test to template engine (ggml-org#19004)

The jinja template parser was missing the 'in' test from
global_builtins(), causing templates using reject("in", ...),
select("in", ...), or 'x is in(y)' to fail with
"selectattr: unknown test 'in'".

This broke tool-calling for Qwen3-Coder and any other model
whose chat template uses the 'in' test.

Added test_is_in supporting array, string, and object containment
checks, mirroring the existing 'in' operator logic in runtime.cpp.

Includes test cases for all three containment types plus
reject/select filter usage.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* reuse test_is_in in binary op

---------

Co-authored-by: Sid Mohan <sidmohan0@users.noreply.github.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
liparetejas pushed a commit to liparetejas/llama.cpp that referenced this pull request Feb 23, 2026
@sidmohan0 @claude
@ngxson @liparetejas
jinja : add missing 'in' test to template engine (ggml-org#19004) (gg…
86b131f 
…ml-org#19239)

* jinja : add missing 'in' test to template engine (ggml-org#19004)

The jinja template parser was missing the 'in' test from
global_builtins(), causing templates using reject("in", ...),
select("in", ...), or 'x is in(y)' to fail with
"selectattr: unknown test 'in'".

This broke tool-calling for Qwen3-Coder and any other model
whose chat template uses the 'in' test.

Added test_is_in supporting array, string, and object containment
checks, mirroring the existing 'in' operator logic in runtime.cpp.

Includes test cases for all three containment types plus
reject/select filter usage.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* reuse test_is_in in binary op

---------

Co-authored-by: Sid Mohan <sidmohan0@users.noreply.github.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
NihilDigit pushed a commit to NihilDigit/llama.cpp that referenced this pull request Apr 12, 2026
@sidmohan0 @ngxson
jinja : add missing 'in' test to template engine (ggml-org#19004) (gg…
0751bc5 
…ml-org#19239)

* jinja : add missing 'in' test to template engine (ggml-org#19004)

The jinja template parser was missing the 'in' test from
global_builtins(), causing templates using reject("in", ...),
select("in", ...), or 'x is in(y)' to fail with
"selectattr: unknown test 'in'".

This broke tool-calling for Qwen3-Coder and any other model
whose chat template uses the 'in' test.

Added test_is_in supporting array, string, and object containment
checks, mirroring the existing 'in' operator logic in runtime.cpp.

Includes test cases for all three containment types plus
reject/select filter usage.


* reuse test_is_in in binary op

---------

Co-authored-by: Sid Mohan <sidmohan0@users.noreply.github.com>
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
Seunghhon pushed a commit to Seunghhon/llama.cpp that referenced this pull request Apr 26, 2026
@sidmohan0 @claude @ngxson
jinja : add missing 'in' test to template engine (ggml-org#19004) (gg…
e4efeca 
…ml-org#19239)

* jinja : add missing 'in' test to template engine (ggml-org#19004)

The jinja template parser was missing the 'in' test from
global_builtins(), causing templates using reject("in", ...),
select("in", ...), or 'x is in(y)' to fail with
"selectattr: unknown test 'in'".

This broke tool-calling for Qwen3-Coder and any other model
whose chat template uses the 'in' test.

Added test_is_in supporting array, string, and object containment
checks, mirroring the existing 'in' operator logic in runtime.cpp.

Includes test cases for all three containment types plus
reject/select filter usage.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* reuse test_is_in in binary op

---------

Co-authored-by: Sid Mohan <sidmohan0@users.noreply.github.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
phuongncn pushed a commit to phuongncn/llama.cpp-gx10-dgx-sparks-deepseekv4 that referenced this pull request Apr 28, 2026
@firecoperana @aldehir
common : introduce composable PEG parser combinators for chat parsing…
ab1d740 
… and new jinja template engine (ggml-org#1369)

---------

Co-authored-by: Piotr Wilkin <piotr.wilkin@syndatis.com>

common : add nemotron 3 parsing (ggml-org#18077)

common : add parser for ministral/mistral large 3/devstral 2 (ggml-org#17713)

common : default content to an empty string (ggml-org#18485)

chat: make tool description and parameters optional per OpenAI spec (ggml-org#18478)

Per the OpenAI API specification, both 'description' and 'parameters'
fields in tool function definitions are optional. Previously, the parser
would throw an exception if these fields were missing.

Attempts to fix ggml-org#17667

common : implement new jinja template engine (ggml-org#18462)
---------

Co-authored-by: Alde Rojas <hello@alde.dev>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

jinja: correct member access rule (ggml-org#18905)

jinja : fix lexing of float literals with sign (ggml-org#18901)

jinja : add missing tojson filter for bool (ggml-org#18900)

jinja : attribute support for join, map and sort (ggml-org#18883)

jinja : fix object item order (and properly implement dictsort) (ggml-org#18904)

tests : add test-jinja -py option for cross-checking (ggml-org#18906)

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

ci : run test-jinja -py on high perf [no ci] (ggml-org#18916)

jinja : fix undefined keys and attributes and int/float as bool (ggml-org#18924)

jinja: support none|string (ggml-org#18995)

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

jinja : implement mixed type object keys (ggml-org#18955)

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>

jinja : undefined should be treated as sequence/iterable (return string/array) by filters/tests (ggml-org#19147)

`tojson` is not a supported `undefined` filter

keep it DRY and fix some types

jinja : do not pass empty tools and add some none filters (ggml-org#19176)

jinja : add unordered_map include to value.h [no ci] (ggml-org#19205)

jinja : add missing 'in' test to template engine (ggml-org#19004) (ggml-org#19239)

The jinja template parser was missing the 'in' test from
global_builtins(), causing templates using reject("in", ...),
select("in", ...), or 'x is in(y)' to fail with
"selectattr: unknown test 'in'".

This broke tool-calling for Qwen3-Coder and any other model
whose chat template uses the 'in' test.

Added test_is_in supporting array, string, and object containment
checks, mirroring the existing 'in' operator logic in runtime.cpp.

Includes test cases for all three containment types plus
reject/select filter usage.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Sid Mohan <sidmohan0@users.noreply.github.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>

Add Jinja support for "indent" string filter (ggml-org#19529)

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

add vendor

refactor chat

server : support preserving reasoning_content in assistant message (ggml-org#18994)

chat : fix translategemma crash on common_chat_format_example (ggml-org#19019)

chat: fix language input for translategemma (ggml-org#19052)

Co-authored-by: Aldehir Rojas <hello@alde.dev>

---------

Co-authored-by: Aldehir Rojas <hello@alde.dev>

chat: fix case where template accepts type content only (ggml-org#19419)

mtmd : chat : Fix extra \n between text and media marker (ggml-org#19595)

Thanks to @tugot17 for detecting and reporting the issue.

For vision models (e.g. LFM2.5-VL-1.6B and Qwen/Qwen3-VL-4B-Instruct) `llama-mtmd-cli` produces identical output to HF implementation.

However `llama-server` doesn't. I traced it down to extra newline
inserted after `<__media__>`.

This happens in `to_json_oaicompat`, that treats media markers as text
and joins all parts with `\n` separator.

PR introduces new type `media_marker` and uses it for media markers.
Extra logic is added to prevent insertion of newlines before and after
media markers.

With this change number of input tokens is identical to HF
implementation and as a result the output is also identical.

I explored other ways to address the issue
* remove completely `\n` between text parts in `to_json_oaicompat`
* merge text messages in server-common.cpp before sending them to `to_json_oaicompat`

Please propose alternative ways of fixing this issue.

Co-authored-by: Piotr Wilkin (ilintar) <piotr.wilkin@syndatis.com>

---------

Co-authored-by: Piotr Wilkin (ilintar) <piotr.wilkin@syndatis.com>

common : merge qwen3-coder and nemotron nano 3 parsers (ggml-org#19765)

common : fix improper trimming in XML parser on complete message (ggml-org#19805)

Co-authored-by: Jules LEIDELINGER <11395311+julio75012@users.noreply.github.com>

jinja: correct stats for tojson and string filters (ggml-org#19785)

jinja : correct default size for string slices (ggml-org#19913)

common : handle unicode during partial json parsing (ggml-org#16526)

common : fix json schema with '\' in literals (ggml-org#17307)

add back qwen_coder_xml and mirothinker

Co-authored-by: Aldehir Rojas <hello@alde.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ngxson ngxson ngxson approved these changes

@ggerganov ggerganov Awaiting requested review from ggerganov ggerganov is a code owner

@CISC CISC Awaiting requested review from CISC CISC is a code owner

@aldehir aldehir Awaiting requested review from aldehir

Assignees

No one assigned

Labels

jinja parser Issues related to the jinja parser testing Everything test related

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Eval bug: Qwen3-Coder template parsing error when tools are enabled

2 participants

@sidmohan0 @ngxson