Overview
We've added a package colossalai.cluster, which provides three utility classes DeviceMeshManager, DistCoordinator and ProcessGroupManager. For most users, they don't need to manage device mesh and process group themselves. Booster and plugins will manage these.
So it's not essential to add tutorials for DeviceMeshManager and ProcessGroupManager. We have to add a tutorial for DistCoordinator.
Overview
We've added a package
colossalai.cluster, which provides three utility classesDeviceMeshManager,DistCoordinatorandProcessGroupManager. For most users, they don't need to manage device mesh and process group themselves. Booster and plugins will manage these.So it's not essential to add tutorials for
DeviceMeshManagerandProcessGroupManager. We have to add a tutorial forDistCoordinator.