[shardformer] merge shardformer to main#4152

Merged

FrankLeeeee merged 49 commits intomainfrom

feature/shardformer

Jul 4, 2023

Commits on Jun 26, 2023

[shardformer] init shardformer code structure (#3731)

FoolPlayer
authored and
FrankLeeeee
committed
[shardformer]: Feature/shardformer, add some docstring and readme (#3816)

FoolPlayer
authored and
FrankLeeeee
committed
[shardformer] updated readme (#3827)
FrankLeeeee
committed
[shardformer] refactored the user api (#3828)
FrankLeeeee
committed
[shardformer] update readme with modules implement doc (#3834)

FoolPlayer
authored and
FrankLeeeee
committed
[shardformer] add Dropout layer support different dropout pattern (#3856)

FoolPlayer
authored and
FrankLeeeee
committed
update README (#3909)

FoolPlayer
authored and
FrankLeeeee
committed
[shardformer] add gpt2 policy and modify shard and slicer to support (#3883)

FoolPlayer
authored and
FrankLeeeee
committed
[shardformer] Align bert value (#3907)

FoolPlayer
authored and
FrankLeeeee
committed
[shardformer] Unit test (#3928)

FoolPlayer
authored and
FrankLeeeee
committed
[shardformer] Add dropout layer in shard model and refactor policy api (#3949)

FoolPlayer
authored and
FrankLeeeee
committed
[shardformer] support llama model using shardformer (#3969)

wukong1992
authored and
FrankLeeeee
committed
[shardformer] shardformer support t5 model (#3994)

wukong1992
authored and
FrankLeeeee
committed
[Shardformer] Downstream bert (#3979)

FoolPlayer
authored and
FrankLeeeee
committed
[shardformer] fix an error in readme (#3988)

FoolPlayer
authored and
FrankLeeeee
committed
[device] support init device mesh from process group (#3990)
FrankLeeeee
committed
[shardformer] Refactor shardformer api (#4001)

FoolPlayer
authored and
FrankLeeeee
committed
[shardformer] integrated linear 1D with dtensor (#3996)
FrankLeeeee
committed
integrate with dist layer (#4011)

FoolPlayer
authored and
FrankLeeeee
committed
[shardformer] refactored embedding and dropout to parallel module (#4013)
FrankLeeeee
committed
[shardformer] removed inplace tensor sharding (#4018)
FrankLeeeee
committed
add vocabembedding layer

FoolPlayer
authored and
FrankLeeeee
committed
support bert with new api

FoolPlayer
authored and
FrankLeeeee
committed
[shardformer] updated doc (#4016)
FrankLeeeee
committed
[shardformer] fix bert and gpt downstream with new api (#4024)

FoolPlayer
authored and
FrankLeeeee
committed
[shardformer] adapted llama to the new API (#4036)
FrankLeeeee
committed
[shardformer] supported T5 and its variants (#4045)
FrankLeeeee
committed
[shardformer] add gpt2 test and layer class refactor (#4041)

FoolPlayer
authored and
FrankLeeeee
committed
[shardformer] adapted T5 and LLaMa test to use kit (#4049)
FrankLeeeee
committed
[shardformer] refactored the shardformer layer structure (#4053)
FrankLeeeee
committed
support kit use for bert/gpt test (#4055)

FoolPlayer
authored and
FrankLeeeee
committed
[shardformer] support module saving and loading (#4062)
FrankLeeeee
committed
[shardformer] add linearconv1d test (#4067)

FoolPlayer
authored and
FrankLeeeee
committed
[shardformer] supported fused qkv checkpoint (#4073)
FrankLeeeee
committed
[shardformer] Add layernorm (#4072)

FoolPlayer
authored and
FrankLeeeee
committed
[test] fixed tests failed due to dtensor change (#4082)
FrankLeeeee
authored
[shardformer] refactored layernorm (#4086)
FrankLeeeee
authored

Commits on Jun 27, 2023

[shardformer] shardformer support opt models (#4091)
flybird11111
authored

Commits on Jun 28, 2023

Commits on Jul 3, 2023

[shardformer] refactored some doc and api (#4137)
FrankLeeeee
authored

Commits on Jul 4, 2023