Maybe the vocab_size can be duplicated to the mainconfig for PEFT to pick up

[lancercat]

transformers/src/transformers/models/gemma3/configuration_gemma3.py

Line 322 in 06c16de

self.vision_config = vision_config

        self.vocab_size=self.text_config.vocab_size;

Due to the following check in peft

https://github.com/huggingface/peft/blob/6c489499300c652a4990cfbcc18539417e73c262/src/peft/utils/save_and_load.py#L249

[zucchini-nlp]

cc @BenjaminBossan , i think we should check for config.get_text_config()

[BenjaminBossan]

Sorry for the late reply, I was out of office. I think I'm missing some context here, I assume you encountered an issue caused by the missing attribute @lancercat?

[lancercat]

Yes. class PeftModel in peft needs this attribute in
_setup_prompt_encoder(self, adapter_name: str).
But i think the have just fixed/hacked it in peft...
https://github.com/huggingface/peft/blob/6d133307adf28e11d061c20aa1f040c1d26d2012/src/peft/peft_model.py#L609

[BenjaminBossan]

Ah thanks, so with this recent change, the original issue is addressed, right?

@zucchini-nlp would it be better to use self.base_model.config.get_text_config() instead of

                if "text_config" in self.base_model.config:
                    vocab_size = self.base_model.config.text_config.vocab_size
                else:
                    vocab_size = self.base_model.config.vocab_size

https://github.com/huggingface/peft/blob/6d133307adf28e11d061c20aa1f040c1d26d2012/src/peft/peft_model.py#L609-L612?

Would this also work with older transformers versions?

[zucchini-nlp]

would it be better to use self.base_model.config.get_text_config() instead of

yes, that would be more general and extra text config under different names. It was added on v4.45.0 it might not work for older versions, not sure since when PEFT has support for that

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.