Enabled Megatron-LM-v1.1.5-ZeRO3-8.3B_param model #13

Merged

jithunnair-amd merged 1 commit intoROCm:masterfrom

rraminen:Megatron-LM-v1.1.5-ZeRO3-8.3B_param

Nov 18, 2021

Commits on Nov 12, 2021

Enabled Megatron-LM-v1.1.5-ZeRO3-8.3B_param model on 8 GPUs and MP_SIZE=1
rraminen
committed