add: controlnet entry to training section in the docs. #2677
Conversation
|
The documentation is not available anymore as the PR was closed or merged. |
patrickvonplaten
left a comment
patrickvonplaten
left a comment
There was a problem hiding this comment.
Thanks that looks nice to me! Going forward once we've added doc testing we should maybe think a bit about how we can test such training pages.
This PR is nice - it mirrors what has been done for dreambooth: https://huggingface.co/docs/diffusers/training/dreambooth
stevhliu
left a comment
stevhliu
left a comment
There was a problem hiding this comment.
Super cool! I love that you included examples from the training after 300 and 600 steps. I think this is really reassuring for users who're following along, and they can compare their outputs to these. 👏
| --> | ||
|
|
||
| # ControlNet | ||
|
|
There was a problem hiding this comment.
It'd be nice to give a short intro (2-3 sentences) describing what ControlNet is.
There was a problem hiding this comment.
I think we can do that for all the training examples we have in a bulk PR.
|
|
||
| [Adding Conditional Control to Text-to-Image Diffusion Models](https://arxiv.org/abs/2302.05543) (ControlNet) by Lvmin Zhang and Maneesh Agrawala. | ||
|
|
||
| This example is based on the [training example in the original ControlNet repository](https://github.com/lllyasviel/ControlNet/blob/main/docs/train.md). It trains a ControlNet to fill circles using a [small synthetic dataset](https://huggingface.co/datasets/fusing/fill50k). |
There was a problem hiding this comment.
| This example is based on the [training example in the original ControlNet repository](https://github.com/lllyasviel/ControlNet/blob/main/docs/train.md). It trains a ControlNet to fill circles using a [small synthetic dataset](https://huggingface.co/datasets/fusing/fill50k). | |
| This guide will show you how to train ControlNet to fill in circles using the [`fill50k`](https://huggingface.co/datasets/fusing/fill50k) dataset. It is based on the [training example](https://github.com/lllyasviel/ControlNet/blob/main/docs/train.md) in the original ControlNet repository. |
There was a problem hiding this comment.
I think it's not actually a guide, though. We are just showing how to use the example script and not discussing the training methodology (for which the script is there).
There was a problem hiding this comment.
I think this is the same thing even if you aren't discussing the training methodology. By showing how to do something, you're still guiding the user. I'm fine with either way though :)
There was a problem hiding this comment.
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
@patrickvonplaten Since the examples are already tested and the training pages are just mirrored copies of those examples, I think it's fine no? |
Yes, it's just that we need to ideally find an automated way of keeping those doc pages in sync with the examples |
You mean mirroring the content of example script READMEs in our docs? |
stevhliu
left a comment
stevhliu
left a comment
There was a problem hiding this comment.
Looks great to me, thanks for all the changes! 👍
|
|
||
| [Adding Conditional Control to Text-to-Image Diffusion Models](https://arxiv.org/abs/2302.05543) (ControlNet) by Lvmin Zhang and Maneesh Agrawala. | ||
|
|
||
| This example is based on the [training example in the original ControlNet repository](https://github.com/lllyasviel/ControlNet/blob/main/docs/train.md). It trains a ControlNet to fill circles using a [small synthetic dataset](https://huggingface.co/datasets/fusing/fill50k). |
There was a problem hiding this comment.
I think this is the same thing even if you aren't discussing the training methodology. By showing how to do something, you're still guiding the user. I'm fine with either way though :)
Because we should.