Skip to content

[tie weights] 🚨 If both weights are present with same weights, still tie them#44497

Merged
Cyrilvallez merged 5 commits intomainfrom
fix-tie-weights
Mar 9, 2026
Merged

[tie weights] 🚨 If both weights are present with same weights, still tie them#44497
Cyrilvallez merged 5 commits intomainfrom
fix-tie-weights

Conversation

@Cyrilvallez
Copy link
Copy Markdown
Member

What does this PR do?

As per the title. Fixes #44466 and avoid issues with torch .bin checkpoints which always contain both keys!

Copy link
Copy Markdown
Contributor

@vasqu vasqu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we all agree to this PR after the internal discussions but let's add a 🚨 just in case. It is slightly breaking, even if it is for the better

@Cyrilvallez Cyrilvallez changed the title [tie weights] If both weights are present with same weights, still tie them [tie weights] 🚨 If both weights are present with same weights, still tie them Mar 9, 2026
@Cyrilvallez Cyrilvallez merged commit 0967791 into main Mar 9, 2026
27 checks passed
@Cyrilvallez Cyrilvallez deleted the fix-tie-weights branch March 9, 2026 15:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[v5] Inconsistent serialization of lm_head.weight (tied weights?) depending on model device in v5/main, while v4.57 behaves correctly

2 participants