Missing docs about how to fine-tune models

[patrickvonplaten]

Describe the bug

Currently there are no docs about how to adapt existing models for fine-tuning. E.g. fine-tuning a text-to-image model for inpainting or other tasks.

It's actually quite easy to initialize a unet from pretrained weights while having a different archicture for some part of the model as shown here: #1619 (comment)

We should put this in the docs, ideally creating a whole "How to adapt a model" doc page

Reproduction

We've now had 4,5 issues about it

Logs

No response

System Info

Not needed

[patrickvonplaten]

Also cc @stevhliu @yiyixuxu @pcuenca in case you have time for this

[stevhliu]

Happy to take this on! :)

[sayakpaul]

@stevhliu please loop me in if you need any help or need to discuss anything :)