Fix gradient checkpointing bugs in freezing part of models (requires_grad=False)#3404

patrickvonplaten merged 5 commits intohuggingface:mainhuggingface/diffusers:mainfrom

IrisRainbowNeko:grad_ckpt_partIrisRainbowNeko/diffusers:grad_ckpt_partCopy head branch name to clipboard