Fix mlcd auto config/model/mapping issues

[ydshieh]

What does this PR do?

It's unclear why the config class has model_type = "mlcd_vision_model" but the model on the hub has "model_type": "mlcd".

This leads to the following failures (load from hub --> save locally --> local locally)

from transformers import MLCDVisionConfig, AutoConfig, AutoImageProcessor, AutoModel

ckpt = "DeepGlint-AI/mlcd-vit-bigG-patch14-336"
local_ckpt = "ckpt_mlcd"

config = AutoConfig.from_pretrained(ckpt)
config.save_pretrained(local_ckpt)
AutoConfig.from_pretrained(local_ckpt)  --> will fail

model = AutoModel.from_pretrained(ckpt)
model.save_pretrained(local_ckpt)
model_local = AutoModel.from_pretrained(local_ckpt) --> will fail too (after comment out the above one)

The error is

The checkpoint you are trying to load has model type mlcd_vision_model but Transformers does not recognize this architecture. ...

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[ydshieh]

This part is in SPECIAL_MODEL_TYPE_TO_MODULE_NAME.

This is also necessary to make the tiny model creation script work for this model

[zucchini-nlp]

can we add a comment why it's needed, because from config code only it's not clear

[ydshieh]

This one is quite clear: it maps the model type to the model module.

If we want to add comment, it's more at

# This is tied to the processing `-` -> `_` in `model_type_to_module_name`. For example, instead of putting
# `transfo-xl` (as in `CONFIG_MAPPING_NAMES`), we should use `transfo_xl`.
DEPRECATED_MODELS = []

SPECIAL_MODEL_TYPE_TO_MODULE_NAME = OrderedDict[str, str](

[zucchini-nlp]

hmm config also has mlcd_vision_model, so prob we don't need mlcd one line above

transformers/src/transformers/models/mlcd/configuration_mlcd.py

Lines 45 to 47 in f074011

	model_type = "mlcd_vision_model"
	base_config_key = "vision_config"

[zucchini-nlp]

unless there are models with both keys saved on the hub

[ydshieh]

the problem is that, on the hub, the model type is "mlcd", like

https://huggingface.co/DeepGlint-AI/mlcd-vit-bigG-patch14-448/blob/main/config.json

https://huggingface.co/DeepGlint-AI/mlcd-vit-bigG-patch14-336/blob/main/config.json

if we remove the "mlcd part", loading from the hub will cause problem I believe.

(but maybe we can add a short comment on the repeated entries ...)

[zucchini-nlp]

Ah, it's the other way round! I am not sure if we can nudge MLCD repo to fix it, so imo we can workaround on our side

[zucchini-nlp]

can we add a comment why it's needed, because from config code only it's not clear

[github-actions]

[For maintainers] Suggested jobs to run (before merge)

run-slow: auto

[github-actions]

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=44730&sha=31a553