Add support for ChatML dataset format in

[philschmid]

What does this PR do?

This PR adds support for a standardized dataset to be automatically formated for training in the SFTTrainer using the apply_chat_template from transformers.
This allow users to pass the dataset without the need of a formatting_func to the SFTTrainer. Example below

from datasets import load_dataset

dataset = load_dataset("philschmid/dolly-15k-oai-style", split="train")

trainer = SFTTrainer(
    model=model,
    args=training_args,
    train_dataset=dataset,
    packing=True,
)

In the init method the SFTTrainer tries to finds the correct formatting function based on the dataset structure. Currently supported datasets are:
- ChatML with [{"role": str, "content": str}]
- instruction with [{"prompt": str, "completion": str}]

Based on the dataset it returns a callable which uses the tokenizer of the model and the corresponding apply_chat_template method. This allows continues fine-tuning for, e.g. Llama2-chat or other models which already have a defined format.

The nice part about is that you can use the "extras" outside of the SFFTrainer, e.g. if you want to format DPO datasets with the methods.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[younesbelkada]

Amazing addition, thanks! I went through the PR and it looks great to me!
Can you add few lines in the documentaiton to explain what it does under the hood? I think the doc section should live inside the SFTTrainer docs - wdyt?

[philschmid]

Can you add few lines in the documentaiton to explain what it does under the hood? I think the doc section should live inside the SFTTrainer docs - wdyt?

I didn't worked on the documentation yet, since i wanted to see what you think. Will work on the docs next.

[younesbelkada]

Looking great to me thanks! I just left one question for the documentation

[younesbelkada]

Maybe worth adding a line saying that the user need to make sure that tokenizer support apply_chat_template - otherwise it'll fail I think no?

[philschmid]

it is always supported. If there is no template defined it falls back to a default template, which is ChatML format from OAI. cc @Rocketknight1 to confirm

[younesbelkada]

ok perfect then!

[Rocketknight1]

Yep, the default format for tokenizers with no chat_template or class-level default_chat_template is ChatML.

[lvwerra]

Thanks a lot @philschmid! Overall very clean, just left a few small nits.

[younesbelkada]

Still looks really good to me! Is it ok for merge @philschmid ?