Skip to content

Add full GGUF loading support for GPT‑OSS (fixes #43366)#45116

Closed
sirzechs66 wants to merge 4 commits intohuggingface:mainfrom
sirzechs66:fix-transformers-issue
Closed

Add full GGUF loading support for GPT‑OSS (fixes #43366)#45116
sirzechs66 wants to merge 4 commits intohuggingface:mainfrom
sirzechs66:fix-transformers-issue

Conversation

@sirzechs66
Copy link
Copy Markdown
Contributor

What does this PR do?

This PR adds full GGUF loading support for GPT‑OSS models (20B/120B). It allows Transformers (and consequently vLLM) to directly load GPT‑OSS GGUF files without falling back to a wrong architecture. The changes include:

  • Architecture registration in GGUF mappings.
  • A custom GptOssTensorProcessor to handle MoE expert splitting and gate/up interleaving.
  • Reconstruction of nested rope_scaling (YaRN) from flat GGUF metadata.
  • Tests: fast registration test + slow integration test using a real 20B GGUF file.

Fixes #43366, supersedes #43757.
Related vLLM issue: vllm-project/vllm#22353

Code Agent Policy

  • I confirm that this is not a pure code agent PR.

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case). – Not applicable, it adds a feature.
  • Did you read the contributor guideline, Pull Request section? – Yes.
  • Was this discussed/approved via a Github issue or the forum? Please add a link to it if that's the case. – Issue GGUF model with architecture gpt-oss support #43366, discussion in comments.
  • Did you make sure to update the documentation with your changes? – Yes!
  • Did you write any new necessary tests? – Yes, GptOssGgufLoadingTest in tests/models/gpt_oss/test_modeling_gpt_oss.py.

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Tagging:

@sirzechs66 sirzechs66 marked this pull request as draft March 30, 2026 12:20
@sirzechs66 sirzechs66 force-pushed the fix-transformers-issue branch from 91529e7 to 5d4515b Compare March 30, 2026 12:48
@github-actions
Copy link
Copy Markdown
Contributor

[For maintainers] Suggested jobs to run (before merge)

run-slow: gpt_oss

@github-actions
Copy link
Copy Markdown
Contributor

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=45116&sha=6e2e65

@sirzechs66 sirzechs66 marked this pull request as ready for review March 30, 2026 13:03
@sirzechs66 sirzechs66 marked this pull request as draft March 30, 2026 13:03
@sirzechs66 sirzechs66 closed this Mar 30, 2026
@sirzechs66 sirzechs66 deleted the fix-transformers-issue branch March 30, 2026 13:04
@sirzechs66
Copy link
Copy Markdown
Contributor Author

i thought the CI conflicts were something from my end , my apologies please feel free to refer to this pr #45118 it still has some issues but i think are from other unmodified files, any help is appretiated

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GGUF model with architecture gpt-oss support

1 participant