[shardforme] integrate dtensor into shardformer #3997

Closed

Assignees

Labels

enhancementshardformer

opened

on Jun 15, 2023

After refactoring the distributed tensor API, we should integrate the DTensor into shardformer to use it for parameter sharding and save/loading checkpoint.

Metadata

Assignees

FrankLeeeee

Labels

enhancementshardformer

Type

No type

Projects

Status

✅ Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests