Skip to content

Add Qwen3.5 support for sequence classification#44406

Merged
zucchini-nlp merged 3 commits intohuggingface:mainfrom
medhakimbedhief:feat/qwen3_5-sequence-classification-support
Mar 4, 2026
Merged

Add Qwen3.5 support for sequence classification#44406
zucchini-nlp merged 3 commits intohuggingface:mainfrom
medhakimbedhief:feat/qwen3_5-sequence-classification-support

Conversation

@medhakimbedhief
Copy link
Copy Markdown
Contributor

@medhakimbedhief medhakimbedhief commented Mar 3, 2026

Adds sequence-classification support for Qwen3.5 in AutoModelForSequenceClassification.

What does this PR do?
This PR enables loading Qwen3.5 checkpoints with AutoModelForSequenceClassification, which previously failed with:
ValueError: Unrecognized configuration class Qwen3_5Config for AutoModelForSequenceClassification.

Changes

Why both mappings?
Qwen3.5 uses a composite VLM config (qwen3_5) with a text sub-config (qwen3_5_text).
Registering both ensures classification works for direct text config usage and composite config loading paths.

Before :
Loading from e.g. Qwen/Qwen3.5-0.8B raises ValueError: Unrecognized configuration class Qwen3_5Config for AutoModelForSequenceClassification.

After this PR:
Loading from Qwen/Qwen3.5-0.8B now resolves to Qwen3_5ForSequenceClassification.

@Cyrilvallez @zucchini-nlp @ArthurZucker

Fixes #44405

- Introduced Qwen3_5ForSequenceClassification class in modeling_qwen3_5.py and modular_qwen3_5.py.
- Updated MODEL_FOR_SEQUENCE_CLASSIFICATION_MAPPING_NAMES to include new Qwen3.5 models.
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 3, 2026

[For maintainers] Suggested jobs to run (before merge)

run-slow: auto, qwen3_5

Copy link
Copy Markdown
Member

@zucchini-nlp zucchini-nlp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR lgtm. We usually don't add a task class for models if there are no weights on the hub for this specific task. Though with the GenericSequenceClassfier I think we can even automatically create sequence classification tasks on top on base models

@medhakimbedhief
Copy link
Copy Markdown
Contributor Author

Just to confirm @zucchini-nlp, are you suggesting we enable sequence classification generically for base checkpoints like Qwen3.5-0.8B-Base via GenericForSequenceClassification, rather than adding a dedicated Qwen3_5ForSequenceClassification class?

That makes sense regarding not expanding the task surface when no task-specific weights exist on the Hub.

@zucchini-nlp
Copy link
Copy Markdown
Member

Ah no, it's fine for this PR. I was just thinking out loud :)

Let's merge then

@zucchini-nlp zucchini-nlp merged commit f60c4e9 into huggingface:main Mar 4, 2026
25 checks passed
@mnbucher
Copy link
Copy Markdown

mnbucher commented Apr 8, 2026

I saw that the current sequence classifier is configured via

config: Qwen3_5TextConfig

https://github.com/huggingface/transformers/pull/44406/changes#diff-403c772331fa104c97f12712903c05e01f6335afdfa1f1719a52ed6ea7d1206dR1875

which seems to ignore the vision tower and is not ideal for VLM-type tasks. is this intended? i'm not very familiar with the whole GenericForSequenceClassification idea etc...

@zucchini-nlp
Copy link
Copy Markdown
Member

nope, kinda related to #44625. Sorry, it got swept off as a lower prio task from my list, will do smth with it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add AutoModelForSequenceClassification support for Qwen3.5 (Qwen3_5Config)

3 participants