Refactor core_model_loading to support FSDP shard-on-read loading#44974
Open
3outeille wants to merge 3 commits intofsdp-vs-ddpfrom
Open
Refactor core_model_loading to support FSDP shard-on-read loading#449743outeille wants to merge 3 commits intofsdp-vs-ddpfrom
3outeille wants to merge 3 commits intofsdp-vs-ddpfrom
Conversation
3outeille
force-pushed
the
fsdp-core-model-loading
branch
from
e5fc7eb to
4b2a921
Compare
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
3outeille
force-pushed
the
fsdp-core-model-loading
branch
from
d48fcc7 to
607cc11
Compare
3outeille
commented
Apr 7, 2026
| return alternation, src_group_to_glob, tgt_group_to_glob | ||
|
|
||
|
|
||
| def resolve_target_wildcards(source_pattern: str, target_pattern: str, source_key: str) -> str: |
| model=model, | ||
| missing_keys=loading_info.missing_keys if loading_info else None, | ||
| ) | ||
| if len(collected_tensors) > 1 and model is not None: |
| # ref.shape is the DTensor global shape. For DTensor-based TP+FSDP, | ||
| # parallelize_module + fully_shard compose correctly, so ref.shape | ||
| # and ref.placements are already correct for the 2D DTensor. | ||
| fsdp_param = DTensor.from_local( |
- DtensorShardOperation for range-math shard-on-read - spawn_materialize() enhancements - from_pretrained wiring for distributed config - Shard operation helpers in tensor_parallel - Shard-on-read and LoadStateDictConfig tests
3outeille
force-pushed
the
fsdp-core-model-loading
branch
from
607cc11 to
739332c
Compare
3outeille
force-pushed
the
fsdp-core-model-loading
branch
from
dbc9619 to
c567240
Compare
3outeille
force-pushed
the
fsdp-core-model-loading
branch
from
c567240 to
c1dab9e
Compare
Contributor
|
View the CircleCI Test Summary for this PR: https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=44974&sha=21f056 |
This was referenced Apr 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TODO:
Running the script from #44996