Fix missing property access for multimodal models

[albertvillanova]

Summary

This PR fixes access to missing attributes for multimodal models in src/liger_kernel/transformers/monkey_patch.py. The main change is to consistently access attributes (like language_model, vision_tower, and visual) through the submodel .model attribute of the parent model, rather than directly from the parent model itself.

This fixes AttributeError after this PR was merged in transformers:

• 🚨 Generalize get_decoder() for multimodal and delete redundant code 🔪  huggingface/transformers#42156

See associated issue in TRL:

• CI fails with dev dependencies: AttributeError: 'Qwen2_5_VLForConditionalGeneration' object has no attribute 'language_model' huggingface/trl#4601

Fix #960.

Details

Fix: Consistent attribute access via .model

• Updated all references to submodules such as language_model, vision_tower, and visual to use the .model attribute (e.g., model.model.language_model instead of model.language_model) across all kernel application functions for models including LLava, Mllama, Gemma3, PaliGemma, Qwen2 VL, Qwen2.5 VL, Qwen3 VL, Qwen3 VL MoE, GLM4V, GLM4V MoE, and InternVL.

Normalization and patching logic updates

• Adjusted normalization and patching calls to operate on submodels accessed via .model, ensuring that layer normalization and RMS normalization are consistently applied to the correct components.

These changes make the codebase more maintainable and robust against future changes in model class implementations.

Testing Done

• Hardware Type:
• run make test to ensure correctness
• run make checkstyle to ensure code style
• run make test-convergence to ensure convergence

[Tcc0403]

Could you help me remove this comment? Thanks!

[albertvillanova]

Done.

[Tcc0403]

We also need to update this condition.

model.model.language for XXXForConditionalGeneration, model.language_model for XXXVLModel

[albertvillanova]

Good catch!

[albertvillanova]

Done.

[Tcc0403]

There still exist some missing attribute error

• MllamaForConditionalGeneration
• Qwen2VLForConditionalGeneration
• Qwen2_5_VLForConditionalGeneration
• InternVLForConditionalGeneration
• Glm4vForConditionalGeneration
• Glm4vMoeForConditionalGeneration

FAILED test/transformers/test_monkey_patch.py::test_apply_liger_kernel_to_instance_for_mllama_for_conditional_generation - AttributeError: 'MllamaForConditionalGeneration' object has no attribute 'language_model'
FAILED test/transformers/test_monkey_patch.py::test_apply_liger_kernel_to_instance_for_qwen2_vl_for_conditional_generation - AttributeError: 'Qwen2VLForConditionalGeneration' object has no attribute 'language_model'
FAILED test/transformers/test_monkey_patch.py::test_apply_liger_kernel_to_instance_for_qwen2_5_vl_for_conditional_generation - AttributeError: 'Qwen2_5_VLForConditionalGeneration' object has no attribute 'language_model'
FAILED test/transformers/test_monkey_patch.py::test_apply_liger_kernel_to_instance_for_internvl - AttributeError: 'InternVLForConditionalGeneration' object has no attribute 'language_model'
FAILED test/transformers/test_monkey_patch.py::test_apply_liger_kernel_to_instance_for_glm4v - AttributeError: 'Glm4vForConditionalGeneration' object has no attribute 'language_model'
FAILED test/transformers/test_monkey_patch.py::test_apply_liger_kernel_to_instance_for_glm4v_moe - AttributeError: 'Glm4vMoeForConditionalGeneration' object has no attribute 'language_model'

Similar errors are also listed in #960 (comment). It's just a reminder for myself, not necessarily have to fix all of them in this PR! We can focus on handling all language_model property in this PR.

[Tcc0403]

Thanks!