main: use jinja chat template system prompt by default#12118
main: use jinja chat template system prompt by default#12118ngxson merged 5 commits intoggml-org:masterfrom
Conversation
From UX perspective, this is very confusing for end-user. I would suggest that we either:
|
Perhaps, but it is already doing this, just wrongly. :)
Makes sense, but maybe in a different PR? |
Yes and it would be better if you we can merge that PR (adding |
Sure, I'll take a look. |
|
Failing tests seems to just be server outage. |
|
I'll test this with Phi-4 a bit later |
|
Ok that works, here is my test command + result (leaving it here for visibility): No system prompt: Incorrect use Correctly using system prompt: |
|
@ngxson I'll make a followup PR to add template formatting in non-conversation mode, and combined use of -sys and -p together. |
* Use jinja chat template system prompt by default * faster conditional order * remove nested ternary --------- Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
* Use jinja chat template system prompt by default * faster conditional order * remove nested ternary --------- Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
Using the new
--jinjaoption would override the builtin system prompt withYou are a helpful assistantif in conversation mode and/or no prompt was defined.Fixed this behaviour
so thatso that the chat template's default is used instead.-poption now either defines system prompt (in conversation mode) or user prompt, and does not fill in a defaultAdds some extra safety checks to be able to process an interactive user prompt with no defined system prompt.
@ochafik PTAL
No system prompt:
Incorrect use
-pCorrectly using system prompt: