Skip to content

Add full GGUF loading support for GPT‑OSS (fixes #43366, supersedes #43757) latest#45506

Merged
ArthurZucker merged 9 commits intohuggingface:mainfrom
sirzechs66:gpt-oss-support-fix
Apr 22, 2026
Merged

Add full GGUF loading support for GPT‑OSS (fixes #43366, supersedes #43757) latest#45506
ArthurZucker merged 9 commits intohuggingface:mainfrom
sirzechs66:gpt-oss-support-fix

Conversation

@sirzechs66
Copy link
Copy Markdown
Contributor

What does this PR do?

This PR adds full GGUF loading support for GPT‑OSS models (20B/120B). It allows Transformers (and consequently vLLM) to directly load GPT‑OSS GGUF files without falling back to a wrong architecture. The changes include:

  • Architecture registration in GGUF mappings.
  • A custom GptOssTensorProcessor to handle MoE expert splitting and gate/up interleaving.
  • Reconstruction of nested rope_scaling (YaRN) from flat GGUF metadata.
  • Tests: fast registration test + slow integration test using a real 20B GGUF file.

Fixes #43366, supersedes #43757.
Related vLLM issue: vllm-project/vllm#22353

Code Agent Policy

  • I confirm that this is not a pure code agent PR.

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case). – Not applicable, it adds a feature.
  • Did you read the contributor guideline, Pull Request section? – Yes.
  • Was this discussed/approved via a Github issue or the forum? Please add a link to it if that's the case. – Issue GGUF model with architecture gpt-oss support #43366, discussion in comments.
  • Did you make sure to update the documentation with your changes? – yes!
  • Did you write any new necessary tests? – Yes, in test/quantization/test_ggml.py

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Tagging:

@sirzechs66
Copy link
Copy Markdown
Contributor Author

@SunMarc please review this

@sirzechs66
Copy link
Copy Markdown
Contributor Author

@ArthurZucker , @Rocketknight1 please review and merge these changes all things are tested and ready

@sirzechs66
Copy link
Copy Markdown
Contributor Author

@SunMarc please review it is ready to be merged

Copy link
Copy Markdown
Member

@SunMarc SunMarc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks ! just a nit

Comment thread docs/source/en/model_doc/gpt_oss.md Outdated
@github-actions
Copy link
Copy Markdown
Contributor

[For maintainers] Suggested jobs to run (before merge)

run-slow: ggml

@ArthurZucker ArthurZucker enabled auto-merge April 22, 2026 06:28
@ArthurZucker ArthurZucker added this pull request to the merge queue Apr 22, 2026
@HuggingFaceDocBuilderDev
Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Merged via the queue into huggingface:main with commit 5e23868 Apr 22, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GGUF model with architecture gpt-oss support

4 participants