🔮 Native VLM support for SFTTrainer

[qgallouedec]

from trl import SFTConfig, SFTTrainer
from datasets import load_dataset

trainer = SFTTrainer(
    model="Qwen/Qwen2.5-VL-3B-Instruct",
    args=SFTConfig(max_length=None),
    train_dataset=load_dataset("trl-lib/llava-instruct-mix", split="train"),
)
trainer.train()

---

all CI failures unrelated

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[merveenoyan]

are we still going to use sft_vlm.py? if so, AutoModelForVision2Seq is no longer a valid class, it's actually not used at all, I think most models are migrated to AutoModelForImageTextToText

[sergiopaniego]

I'd aim to remove them all if possible.
Second part is covered in #3836 😄

[merveenoyan]

PR is good, also tested on multiple models

only me running on accelerate didn't work well but it might be me lol