Add GGUF support to Gemma4 (31B & 26B-A4B) text

[UsamaKenway]

What does this PR do?

Adds support for Gemma4 GGUF models : 26B Moe and 31B dense model.
This also helps me load GGUF model in vLLM that I'm also working on.
ViT part is not included

Code Agent Policy

• I confirm that this is not a pure code agent PR.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case. Related to Community contribution: Adding GGUF support for more architectures #33260.
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?

Tests:

# 1. Gemma4 31B (q4_k_m)
(py312venv) usamakenway@Megatron: RUN_SLOW=1 pytest tests/quantization/ggml/test_ggml.py::GgufModelTests::test_gemma4_31b_q4_k_m -s 
tests/quantization/ggml/test_ggml.py::GgufModelTests::test_gemma4_31b_q4_k_m . [100%]
================================= 1 passed in 498.43s (0:08:18) =================================

# 2. Gemma4 31B IT (q8_0)
(py312venv) usamakenway@Megatron: RUN_SLOW=1 pytest tests/quantization/ggml/test_ggml.py::GgufModelTests::test_gemma4_31b_it_q8_0 -s
tests/quantization/ggml/test_ggml.py::GgufModelTests::test_gemma4_31b_it_q8_0 . [100%]
================================= 1 passed in 506.82s (0:08:26) =================================

# 3. Gemma4 31B IT (q4_k_m)
(py312venv) usamakenway@Megatron: RUN_SLOW=1 pytest tests/quantization/ggml/test_ggml.py::GgufModelTests::test_gemma4_31b_it_q4_k_m -s
tests/quantization/ggml/test_ggml.py::GgufModelTests::test_gemma4_31b_it_q4_k_m . [100%]
================================= 1 passed in 488.88s (0:08:08) =================================

# 4. Gemma4 26B IT (q8_0)
(py312venv) usamakenway@Megatron: RUN_SLOW=1 pytest tests/quantization/ggml/test_ggml.py::GgufModelTests::test_gemma4_26b_it_q8_0 -s
tests/quantization/ggml/test_ggml.py::GgufModelTests::test_gemma4_26b_it_q8_0 . [100%]
================================= 1 passed in 420.12s (0:07:00) =================================

# 5. Gemma4 26B IT (q4_k_m)
(py312venv) usamakenway@Megatron: RUN_SLOW=1 pytest tests/quantization/ggml/test_ggml.py::GgufModelTests::test_gemma4_26b_it_q4_k_m -s
tests/quantization/ggml/test_ggml.py::GgufModelTests::test_gemma4_26b_it_q4_k_m . [100%]
================================= 1 passed in 473.56s (0:07:53) =================================

I saw repeated hello for 31B in unittest so i thought to run this output to confirm if chat is working fine:

# === Test 1: Bare text (unit test style) ===
text = tokenizer(self.example_text, return_tensors="pt")["input_ids"]
out = model.generate(text, max_new_tokens=10)
# Output: "HelloKelloKelloKelloKelloKello"

# === Test 2: Chat completion ===
messages = [{"role": "user", "content": "Hi how are you"}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_dict=True, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=30, do_sample=False)
# Output: "I'm doing well, thank you for asking! How are you doing today? Is there anything I can help you with?"
an help you with?<turn|>
<|channel>thought

# === Test 3: Sentence continuation ===
                                                                                                                                                                                                                                                                                                                        
text = tokenizer("The capital of France is", return_tensors="pt")["input_ids"].to(model.device)                                                                                                                                                                                                                                                                            
print(f"input_ids: {text}")                                                                                                                                                                                                                                                                                                                                                
out = model.generate(text, max_new_tokens=20, do_sample=False)                                                                                                                                                                                                                                                                                                             
print(f"Output: {tokenizer.decode(out[0], skip_special_tokens=True)}")        

input_ids: tensor([[   2,  818, 5279,  529, 7001,  563]], device='cuda:0')
Output: The capital of France is Paris.<turn|>
<|channel>thought
<channel|>The capital of France is Paris.<turn|>}<turn|>

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[Rocketknight1]

cc @SunMarc

[SunMarc]

Thanks, left a comment

[UsamaKenway]

Addressed the feedback regarding expected values and updated the tests, adding base model.

[github-actions]

[For maintainers] Suggested jobs to run (before merge)

run-slow: ggml

[SunMarc]

Thanks, just a nit

[SunMarc]

our CI won't have enough space to run these models. So let's just skip those for now