AttributeError for 'language_model' in transformers v5 #960
Description
🐛 Describe the bug
The function _apply_liger_kernel_to_instance raises AttributeError (no attribute 'language_model') for transformers v5 (current main branch):
AttributeError: 'Qwen2_5_VLForConditionalGeneration' object has no attribute 'language_model'
_apply_liger_kernel_to_instance(model=model, **kernel_config)
.venv/lib/python3.12/site-packages/liger_kernel/transformers/monkey_patch.py:2851: in _apply_liger_kernel_to_instance
apply_fn(model=model, **applicable_kwargs)
.venv/lib/python3.12/site-packages/liger_kernel/transformers/monkey_patch.py:1619: in apply_liger_kernel_to_qwen2_5_vl
text_model: Qwen2_5_VLTextModel = model.language_model
^^^^^^^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = Qwen2_5_VLForConditionalGeneration(
(model): Qwen2_5_VLModel(
(visual): Qwen2_5_VisionTransformerPretrainedModel...tary_emb): Qwen2_5_VLRotaryEmbedding()
)
)
(lm_head): Linear(in_features=16, out_features=151936, bias=False)
)
name = 'language_model'
def __getattr__(self, name: str) -> Union[Tensor, "Module"]:
if "_parameters" in self.__dict__:
_parameters = self.__dict__["_parameters"]
if name in _parameters:
return _parameters[name]
if "_buffers" in self.__dict__:
_buffers = self.__dict__["_buffers"]
if name in _buffers:
return _buffers[name]
if "_modules" in self.__dict__:
modules = self.__dict__["_modules"]
if name in modules:
return modules[name]
> raise AttributeError(
f"'{type(self).__name__}' object has no attribute '{name}'"
)
E AttributeError: 'Qwen2_5_VLForConditionalGeneration' object has no attribute 'language_model'
.venv/lib/python3.12/site-packages/torch/nn/modules/module.py:1964: AttributeErrorSee associated issue downstream in trl:
Reproduce
No response
Versions
Operating System: Linux-5.15.0-1048-aws-x86_64-with-glibc2.31
Python version: 3.10.18
Liger Kernel version: 0.6.4
PyTorch version: 2.8.0+cu128
CUDA version: 12.8
HIP(ROCm) version: Not available
Triton version: 3.4.0
Transformers version: 5.0.0.dev0
XPU version: XPU Not Available