diff --git a/docs/source/en/training/controlnet.mdx b/docs/source/en/training/controlnet.mdx index 7a5454107b83..94e3d969b80a 100644 --- a/docs/source/en/training/controlnet.mdx +++ b/docs/source/en/training/controlnet.mdx @@ -74,6 +74,7 @@ wget https://huggingface.co/datasets/huggingface/documentation-images/resolve/ma wget https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/controlnet_training/conditioning_image_2.png ``` +Specify the `MODEL_NAME` environment variable (either a Hub model repository id or a path to the directory containing the model weights) and pass it to the [`~diffusers.DiffusionPipeline.from_pretrained.pretrained_model_name_or_path`] argument. ```bash export MODEL_DIR="runwayml/stable-diffusion-v1-5" diff --git a/docs/source/en/training/dreambooth.mdx b/docs/source/en/training/dreambooth.mdx index 88ded0e009dc..c5a5a047d114 100644 --- a/docs/source/en/training/dreambooth.mdx +++ b/docs/source/en/training/dreambooth.mdx @@ -50,6 +50,20 @@ from accelerate.utils import write_basic_config write_basic_config() ``` +Finally, download a [few images of a dog](https://huggingface.co/datasets/diffusers/dog-example) to DreamBooth with: + +```py +from huggingface_hub import snapshot_download + +local_dir = "./dog" +snapshot_download( + "diffusers/dog-example", + local_dir=local_dir, + repo_type="dataset", + ignore_patterns=".gitattributes", +) +``` + ## Finetuning @@ -60,22 +74,13 @@ DreamBooth finetuning is very sensitive to hyperparameters and easy to overfit. -Let's try DreamBooth with a -[few images of a dog](https://huggingface.co/datasets/diffusers/dog-example); -download and save them to a directory and then set the `INSTANCE_DIR` environment variable to that path: +Set the `INSTANCE_DIR` environment variable to the path of the directory containing the dog images. -```python -local_dir = "./path_to_training_images" -snapshot_download( - "diffusers/dog-example", - local_dir=local_dir, repo_type="dataset", - ignore_patterns=".gitattributes", -) -``` +Specify the `MODEL_NAME` environment variable (either a Hub model repository id or a path to the directory containing the model weights) and pass it to the [`~diffusers.DiffusionPipeline.from_pretrained.pretrained_model_name_or_path`] argument. ```bash export MODEL_NAME="CompVis/stable-diffusion-v1-4" -export INSTANCE_DIR="path_to_training_images" +export INSTANCE_DIR="./dog" export OUTPUT_DIR="path_to_saved_model" ``` @@ -105,11 +110,13 @@ Before running the script, make sure you have the requirements installed: pip install -U -r requirements.txt ``` +Specify the `MODEL_NAME` environment variable (either a Hub model repository id or a path to the directory containing the model weights) and pass it to the [`~diffusers.DiffusionPipeline.from_pretrained.pretrained_model_name_or_path`] argument. + Now you can launch the training script with the following command: ```bash export MODEL_NAME="duongna/stable-diffusion-v1-4-flax" -export INSTANCE_DIR="path-to-instance-images" +export INSTANCE_DIR="./dog" export OUTPUT_DIR="path-to-save-model" python train_dreambooth_flax.py \ @@ -135,7 +142,7 @@ The authors recommend generating `num_epochs * num_samples` images for prior pre ```bash export MODEL_NAME="CompVis/stable-diffusion-v1-4" -export INSTANCE_DIR="path_to_training_images" +export INSTANCE_DIR="./dog" export CLASS_DIR="path_to_class_images" export OUTPUT_DIR="path_to_saved_model" @@ -160,7 +167,7 @@ accelerate launch train_dreambooth.py \ ```bash export MODEL_NAME="duongna/stable-diffusion-v1-4-flax" -export INSTANCE_DIR="path-to-instance-images" +export INSTANCE_DIR="./dog" export CLASS_DIR="path-to-class-images" export OUTPUT_DIR="path-to-save-model" @@ -197,7 +204,7 @@ Pass the `--train_text_encoder` argument to the training script to enable finetu ```bash export MODEL_NAME="CompVis/stable-diffusion-v1-4" -export INSTANCE_DIR="path_to_training_images" +export INSTANCE_DIR="./dog" export CLASS_DIR="path_to_class_images" export OUTPUT_DIR="path_to_saved_model" @@ -224,7 +231,7 @@ accelerate launch train_dreambooth.py \ ```bash export MODEL_NAME="duongna/stable-diffusion-v1-4-flax" -export INSTANCE_DIR="path-to-instance-images" +export INSTANCE_DIR="./dog" export CLASS_DIR="path-to-class-images" export OUTPUT_DIR="path-to-save-model" @@ -360,7 +367,7 @@ Then pass the `--use_8bit_adam` option to the training script: ```bash export MODEL_NAME="CompVis/stable-diffusion-v1-4" -export INSTANCE_DIR="path_to_training_images" +export INSTANCE_DIR="./dog" export CLASS_DIR="path_to_class_images" export OUTPUT_DIR="path_to_saved_model" @@ -389,7 +396,7 @@ To run DreamBooth on a 12GB GPU, you'll need to enable gradient checkpointing, t ```bash export MODEL_NAME="CompVis/stable-diffusion-v1-4" -export INSTANCE_DIR="path-to-instance-images" +export INSTANCE_DIR="./dog" export CLASS_DIR="path-to-class-images" export OUTPUT_DIR="path-to-save-model" @@ -436,7 +443,7 @@ Launch training with the following command: ```bash export MODEL_NAME="CompVis/stable-diffusion-v1-4" -export INSTANCE_DIR="path_to_training_images" +export INSTANCE_DIR="./dog" export CLASS_DIR="path_to_class_images" export OUTPUT_DIR="path_to_saved_model" diff --git a/docs/source/en/training/instructpix2pix.mdx b/docs/source/en/training/instructpix2pix.mdx index c485db6d6b20..ff34ec335656 100644 --- a/docs/source/en/training/instructpix2pix.mdx +++ b/docs/source/en/training/instructpix2pix.mdx @@ -74,8 +74,7 @@ write_basic_config() As mentioned before, we'll use a [small toy dataset](https://huggingface.co/datasets/fusing/instructpix2pix-1000-samples) for training. The dataset is a smaller version of the [original dataset](https://huggingface.co/datasets/timbrooks/instructpix2pix-clip-filtered) used in the InstructPix2Pix paper. -Configure environment variables such as the dataset identifier and the Stable Diffusion -checkpoint: +Specify the `MODEL_NAME` environment variable (either a Hub model repository id or a path to the directory containing the model weights) and pass it to the [`~diffusers.DiffusionPipeline.from_pretrained.pretrained_model_name_or_path`] argument. You'll also need to specify the dataset name in `DATASET_ID`: ```bash export MODEL_NAME="runwayml/stable-diffusion-v1-5" diff --git a/docs/source/en/training/lora.mdx b/docs/source/en/training/lora.mdx index ac2311df9f1e..7e3c3c0b2b68 100644 --- a/docs/source/en/training/lora.mdx +++ b/docs/source/en/training/lora.mdx @@ -52,7 +52,9 @@ Finetuning a model like Stable Diffusion, which has billions of parameters, can Let's finetune [`stable-diffusion-v1-5`](https://huggingface.co/runwayml/stable-diffusion-v1-5) on the [Pokémon BLIP captions](https://huggingface.co/datasets/lambdalabs/pokemon-blip-captions) dataset to generate your own Pokémon. -To start, make sure you have the `MODEL_NAME` and `DATASET_NAME` environment variables set. The `OUTPUT_DIR` and `HUB_MODEL_ID` variables are optional and specify where to save the model to on the Hub: +Specify the `MODEL_NAME` environment variable (either a Hub model repository id or a path to the directory containing the model weights) and pass it to the [`~diffusers.DiffusionPipeline.from_pretrained.pretrained_model_name_or_path`] argument. You'll also need to set the `DATASET_NAME` environment variable to the name of the dataset you want to train on. + +The `OUTPUT_DIR` and `HUB_MODEL_ID` variables are optional and specify where to save the model to on the Hub: ```bash export MODEL_NAME="runwayml/stable-diffusion-v1-5" @@ -140,7 +142,9 @@ Load the LoRA weights from your finetuned model *on top of the base model weight Let's finetune [`stable-diffusion-v1-5`](https://huggingface.co/runwayml/stable-diffusion-v1-5) with DreamBooth and LoRA with some 🐶 [dog images](https://drive.google.com/drive/folders/1BO_dyz-p65qhBRRMRA4TbZ8qW4rB99JZ). Download and save these images to a directory. -To start, make sure you have the `MODEL_NAME` and `INSTANCE_DIR` (path to directory containing images) environment variables set. The `OUTPUT_DIR` variables is optional and specifies where to save the model to on the Hub: +To start, specify the `MODEL_NAME` environment variable (either a Hub model repository id or a path to the directory containing the model weights) and pass it to the [`~diffusers.DiffusionPipeline.from_pretrained.pretrained_model_name_or_path`] argument. You'll also need to set `INSTANCE_DIR` to the path of the directory containing the images. + +The `OUTPUT_DIR` variables is optional and specifies where to save the model to on the Hub: ```bash export MODEL_NAME="runwayml/stable-diffusion-v1-5" diff --git a/docs/source/en/training/text2image.mdx b/docs/source/en/training/text2image.mdx index 70f8c003a787..dabb68397f78 100644 --- a/docs/source/en/training/text2image.mdx +++ b/docs/source/en/training/text2image.mdx @@ -72,7 +72,9 @@ To load a checkpoint to resume training, pass the argument `--resume_from_checkp -Launch the [PyTorch training script](https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/train_text_to_image.py) for a fine-tuning run on the [Pokémon BLIP captions](https://huggingface.co/datasets/lambdalabs/pokemon-blip-captions) dataset like this: +Launch the [PyTorch training script](https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/train_text_to_image.py) for a fine-tuning run on the [Pokémon BLIP captions](https://huggingface.co/datasets/lambdalabs/pokemon-blip-captions) dataset like this. + +Specify the `MODEL_NAME` environment variable (either a Hub model repository id or a path to the directory containing the model weights) and pass it to the [`~diffusers.DiffusionPipeline.from_pretrained.pretrained_model_name_or_path`] argument. {"path": "../../../../examples/text_to_image/README.md", @@ -141,6 +143,8 @@ Before running the script, make sure you have the requirements installed: pip install -U -r requirements_flax.txt ``` +Specify the `MODEL_NAME` environment variable (either a Hub model repository id or a path to the directory containing the model weights) and pass it to the [`~diffusers.DiffusionPipeline.from_pretrained.pretrained_model_name_or_path`] argument. + Now you can launch the [Flax training script](https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/train_text_to_image_flax.py) like this: ```bash diff --git a/docs/source/en/training/text_inversion.mdx b/docs/source/en/training/text_inversion.mdx index 6e6971d7f119..e47a0519c704 100644 --- a/docs/source/en/training/text_inversion.mdx +++ b/docs/source/en/training/text_inversion.mdx @@ -1,4 +1,4 @@ -