[shardformer] update hybrid parallel plugin and fix bugs#4612

Merged

ver217 merged 35 commits intomainfrom

feature/shardformer

Sep 5, 2023

Commits on Aug 16, 2023

Commits on Aug 21, 2023

[shardformer] support tp+zero for shardformer (#4472)
Baizhou Zhang
authored

Commits on Aug 22, 2023

rename chatglm to chatglm2 (#4484)
CjhHa1
authored
[shardformer/sequence parallel] not support opt of seq-parallel, add warning and fix a bug in gpt2 pp (#4488)
FoolPlayer
authored
[shardformer] chatglm support sequence parallel (#4482)
flybird11111
authored

Commits on Aug 23, 2023

[shardformer] tests for 3d parallel (#4493)
CjhHa1
authored

Commits on Aug 24, 2023

[shardformer] vit/llama/t5 ignore the sequence parallelism flag and some fix. (#4498)
flybird11111
authored

Commits on Aug 25, 2023

Commits on Aug 28, 2023

Commits on Aug 29, 2023

Commits on Aug 30, 2023

Commits on Aug 31, 2023

[shardformer] fix submodule replacement bug when enabling pp (#4544)
Baizhou Zhang
authored
[shardformer] support sharded optimizer checkpointIO of HybridParallelPlugin (#4540)
Baizhou Zhang
authored

Commits on Sep 1, 2023

Commits on Sep 4, 2023