[Misc] add enable_thinking to template kwargs

[JJJYmmm]

What does this PR do?

Fix QwenLM/Qwen3.6#97.
This PR adds enable_thinking to the chat-template kwargs. With this change, enable_thinking is treated as a template-level argument in the tokenize=True path, so it no longer leaks into processor call kwargs and triggers the “not a valid argument ... ignored” warning, while preserving existing thinking behavior.

[Rocketknight1]

Ah, this is interesting! cc @zucchini-nlp, the underlying problem here is that chat templates can accept arbitrary kwargs, but we don't have a good way of filtering those arbitrary kwargs out for e.g. tokenization. We probably want a more general solution than manually adding specific template kwargs to the list, right?

[zucchini-nlp]

Yep, also started noticing it after mistral4 release and pushing a proper PR today (if I have time) or tomorrow first priority. The root cause is as Matt said, because we used to call tokenizer.apply_chat_template in the past and needed to wrokaround between tokenizer vs processor kwargs

I'd prefer to not merge this PR yet

[JJJYmmm]

Agree! That makes sense. In my latest commit, I use jinja2.meta.find_undeclared_variables to auto-detect template kwargs, which may help with this more generally.

[github-actions]

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=44817&sha=fd3850

[zucchini-nlp]

Nice way for us to detect vars and support BC, thanks

[JJJYmmm]

Closing this pr in favor of #44881. 🥂