convert : add Llama4ForCausalLM

[ngxson]

Fix #16021

Tested with:

• https://huggingface.co/facebook/MobileLLM-R1-140M
• https://huggingface.co/facebook/MobileLLM-R1-950M

Very important note from model card:

These models are not general-purpose chat models. They are Supervised Fine-Tuned (SFT) models, specifically trained to address mathematical, programming (Python, C++), and scientific problems.

That is the reason why model cannot response to simple "hi"

[CISC]

LGTM

[CISC]

Usually there's a use_qk_norm config.json value btw, is that not the case for all Llama4 based models?

[ngxson]

Usually there's a use_qk_norm config.json value btw, is that not the case for all Llama4 based models?

It's the case for all llama 4 models, but traditionally this the KQ norm is something arch-specific so I didn't add GGUF metadata in the first place. but yes I think we should add it now.

[ngxson]

Hmm but as this may affect existing GGUF for the larger llama 4 MoE, I think it will be a bit messy to add it as GGUF metadata.

In anyway, the rule is currently as follow: all llama 4 models uses KQ norm except for the biggest 17B_128E

So I think we can keep it as-is

[CISC]

Hmm but as this may affect existing GGUF for the larger llama 4 MoE, I think it will be a bit messy to add it as GGUF metadata.

In anyway, the rule is currently as follow: all llama 4 models uses KQ norm except for the biggest 17B_128E

So I think we can keep it as-is

Absolutely, just wondering if it's worth adding it for the future.