🚨 [ALM] Add base model without head

[eustlb]

What does this PR do?

This is motivated to simplify changes required for vLLM compatibility as seen in #39330

Fix a discrepancy in ALMs design compared to VLMs that for most of them don't have a base model class because the use the causal model for the text model directly, while Llava style (which is indeed more aligned with the lib philosophy) is rather:
XxModel: encoder + projector + language base model backbone
XxForConditionalGeneration: XxModel + lm_head

why do we actually have this discrepancy?

qwen2_audio was the first ALM in transformers, modelling reflects the status of llava when it was added
all the ALMs merged since then replicated this.

llava and VLMs evolved thanks to

• 🔴 [VLM] Add base model without head  #37033
• Allow VLMs to have a correct base_model #41589
• 🚨 Generalize get_decoder() for multimodal and delete redundant code 🔪  #42156

how do we ensure BC

• weight loading: thanks to the dynamic weight loader and conversion mapping
• accessing model.language_model
• auto-mapping → this is not BC, and it is not avoidable 🚨

diff with how it's been done for VLM

I'd rather use the introduced @forward_base_model_attrs that avoid having to add as done for VLMs

@property                                                                                                
def language_model(self):                                                                                                                                                                                                        
    return self.model.language_model

clearer IMO and avoid forgetting in the future why such a thing was added in the first place.

🚨🚨🚨

We don't fully ensure BC, this is breaking:

• AutoModel.from_pretrained("Qwen/Qwen2-Audio-7B") now returns Qwen2AudioModel, not Qwen2AudioForConditionalGeneration, some for other ALMs 🚨
• likewise model.language_model is still returned but it now returns base model and not the causal one as before 🚨

TODO

• add base_model_class in the ALM tester when audio tester class #45391 is merged

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.