[Feature] MoE refactor with newest version of ZeRO#5801

Merged

botbw merged 1 commit intohpcaitech:feature/moefrom

Hz188:feature/moe

Jun 12, 2024

Commits on Jun 12, 2024

[moe refactor] update unit test with the refactored ZeRO and remove useless test
Hz188
committed