Support distributed Adam with T5 and support overlapped grad reductions with pipeline parallelism#4900
Merged
ericharper merged 30 commits intoNVIDIA-NeMo:mainfrom Oct 21, 2022
Commits
Commits on Aug 23, 2022
Commits on Aug 25, 2022
Commits on Aug 26, 2022
Commits on Aug 29, 2022
Commits on Sep 7, 2022
Commits on Sep 8, 2022
Commits on Sep 9, 2022
Commits on Sep 15, 2022
Commits on Sep 20, 2022
Commits on Oct 4, 2022
Commits on Oct 11, 2022
![pre-commit-ci[bot]](https://avatars.githubusercontent.com/in/68672?v=4&size=32){kind=avatar}
Commits on Oct 20, 2022
- committed
- committed
- authored
- committed
- authored
