We keep getting issues where community members want to know how to do multi-GPU training with our training examples. An example: #2981.
I think we should test the following examples on a priority (as for others, multi-GPU is probably overkill) and ensure their respective READMEs have a short section on multi-GPU support (as suggested by @patrickvonplaten here):
- Text-to-image
- ControlNet
- InstructPix2Pix (being introduced here)
DreamBooth and Textual Inversion won't probably benefit from multi-GPU setup that much, so okay for me to not focus on them at the moment.
I will test text-to-image as well as InstructPix2Pix. @williamberman or @yiyixuxu could do it for ControlNet?