[ShardFormer]: Bert fine tune example with hybrid parallel plugin #4566

Closed

ppt0011 wants to merge 3 commits intohpcaitech:feature/shardformerfrom

ppt0011:feature/shardformer

Commits on Aug 31, 2023

[shardformer] bert finetune with hybrid parallel plugin
ppt0011
committed
Merge branch 'feature/shardformer' of https://github.com/ppt0011/ColossalAI into shardformer
ppt0011
committed
[shardformer] fix model and optimizer correspondance for pp+zero1
ppt0011
committed