Improve communication overlapping in FP8 distributed optimizer#8221
Merged
timmoon10 merged 14 commits intoNVIDIA-NeMo:mainfrom Feb 8, 2024
Merged
Improve communication overlapping in FP8 distributed optimizer#8221timmoon10 merged 14 commits intoNVIDIA-NeMo:mainfrom
timmoon10 merged 14 commits intoNVIDIA-NeMo:mainfrom
![pre-commit-ci[bot]](https://avatars.githubusercontent.com/in/68672?v=4&size=32){kind=avatar}