From 69e000d2bcaa84f63da63191431b97298c851a36 Mon Sep 17 00:00:00 2001 From: Steven Liu Date: Tue, 18 Apr 2023 11:08:17 -0400 Subject: [PATCH 1/2] clarify training arg --- docs/source/en/training/controlnet.mdx | 1 + docs/source/en/training/dreambooth.mdx | 6 +++++- docs/source/en/training/instructpix2pix.mdx | 3 +-- docs/source/en/training/lora.mdx | 8 ++++++-- docs/source/en/training/text2image.mdx | 6 +++++- docs/source/en/training/text_inversion.mdx | 6 +++++- 6 files changed, 23 insertions(+), 7 deletions(-) diff --git a/docs/source/en/training/controlnet.mdx b/docs/source/en/training/controlnet.mdx index 7a5454107b83..94e3d969b80a 100644 --- a/docs/source/en/training/controlnet.mdx +++ b/docs/source/en/training/controlnet.mdx @@ -74,6 +74,7 @@ wget https://huggingface.co/datasets/huggingface/documentation-images/resolve/ma wget https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/controlnet_training/conditioning_image_2.png ``` +Specify the `MODEL_NAME` environment variable (either a Hub model repository id or a path to the directory containing the model weights) and pass it to the [`~diffusers.DiffusionPipeline.from_pretrained.pretrained_model_name_or_path`] argument. ```bash export MODEL_DIR="runwayml/stable-diffusion-v1-5" diff --git a/docs/source/en/training/dreambooth.mdx b/docs/source/en/training/dreambooth.mdx index 908355e496dc..7be5b2a6a248 100644 --- a/docs/source/en/training/dreambooth.mdx +++ b/docs/source/en/training/dreambooth.mdx @@ -60,7 +60,9 @@ DreamBooth finetuning is very sensitive to hyperparameters and easy to overfit. -Let's try DreamBooth with a [few images of a dog](https://drive.google.com/drive/folders/1BO_dyz-p65qhBRRMRA4TbZ8qW4rB99JZ); download and save them to a directory and then set the `INSTANCE_DIR` environment variable to that path: +Let's try DreamBooth with a [few images of a dog](https://drive.google.com/drive/folders/1BO_dyz-p65qhBRRMRA4TbZ8qW4rB99JZ); download and save them to a directory and then set the `INSTANCE_DIR` environment variable to that path. + +Specify the `MODEL_NAME` environment variable (either a Hub model repository id or a path to the directory containing the model weights) and pass it to the [`~diffusers.DiffusionPipeline.from_pretrained.pretrained_model_name_or_path`] argument. ```bash export MODEL_NAME="CompVis/stable-diffusion-v1-4" @@ -94,6 +96,8 @@ Before running the script, make sure you have the requirements installed: pip install -U -r requirements.txt ``` +Specify the `MODEL_NAME` environment variable (either a Hub model repository id or a path to the directory containing the model weights) and pass it to the [`~diffusers.DiffusionPipeline.from_pretrained.pretrained_model_name_or_path`] argument. + Now you can launch the training script with the following command: ```bash diff --git a/docs/source/en/training/instructpix2pix.mdx b/docs/source/en/training/instructpix2pix.mdx index c485db6d6b20..ff34ec335656 100644 --- a/docs/source/en/training/instructpix2pix.mdx +++ b/docs/source/en/training/instructpix2pix.mdx @@ -74,8 +74,7 @@ write_basic_config() As mentioned before, we'll use a [small toy dataset](https://huggingface.co/datasets/fusing/instructpix2pix-1000-samples) for training. The dataset is a smaller version of the [original dataset](https://huggingface.co/datasets/timbrooks/instructpix2pix-clip-filtered) used in the InstructPix2Pix paper. -Configure environment variables such as the dataset identifier and the Stable Diffusion -checkpoint: +Specify the `MODEL_NAME` environment variable (either a Hub model repository id or a path to the directory containing the model weights) and pass it to the [`~diffusers.DiffusionPipeline.from_pretrained.pretrained_model_name_or_path`] argument. You'll also need to specify the dataset name in `DATASET_ID`: ```bash export MODEL_NAME="runwayml/stable-diffusion-v1-5" diff --git a/docs/source/en/training/lora.mdx b/docs/source/en/training/lora.mdx index 1c72fbbc8d58..d221b00fc755 100644 --- a/docs/source/en/training/lora.mdx +++ b/docs/source/en/training/lora.mdx @@ -50,7 +50,9 @@ Finetuning a model like Stable Diffusion, which has billions of parameters, can Let's finetune [`stable-diffusion-v1-5`](https://huggingface.co/runwayml/stable-diffusion-v1-5) on the [Pokémon BLIP captions](https://huggingface.co/datasets/lambdalabs/pokemon-blip-captions) dataset to generate your own Pokémon. -To start, make sure you have the `MODEL_NAME` and `DATASET_NAME` environment variables set. The `OUTPUT_DIR` and `HUB_MODEL_ID` variables are optional and specify where to save the model to on the Hub: +Specify the `MODEL_NAME` environment variable (either a Hub model repository id or a path to the directory containing the model weights) and pass it to the [`~diffusers.DiffusionPipeline.from_pretrained.pretrained_model_name_or_path`] argument. You'll also need to set the `DATASET_NAME` environment variable to the name of the dataset you want to train on. + +The `OUTPUT_DIR` and `HUB_MODEL_ID` variables are optional and specify where to save the model to on the Hub: ```bash export MODEL_NAME="runwayml/stable-diffusion-v1-5" @@ -138,7 +140,9 @@ Load the LoRA weights from your finetuned model *on top of the base model weight Let's finetune [`stable-diffusion-v1-5`](https://huggingface.co/runwayml/stable-diffusion-v1-5) with DreamBooth and LoRA with some 🐶 [dog images](https://drive.google.com/drive/folders/1BO_dyz-p65qhBRRMRA4TbZ8qW4rB99JZ). Download and save these images to a directory. -To start, make sure you have the `MODEL_NAME` and `INSTANCE_DIR` (path to directory containing images) environment variables set. The `OUTPUT_DIR` variables is optional and specifies where to save the model to on the Hub: +To start, specify the `MODEL_NAME` environment variable (either a Hub model repository id or a path to the directory containing the model weights) and pass it to the [`~diffusers.DiffusionPipeline.from_pretrained.pretrained_model_name_or_path`] argument. You'll also need to set `INSTANCE_DIR` to the path of the directory containing the images. + +The `OUTPUT_DIR` variables is optional and specifies where to save the model to on the Hub: ```bash export MODEL_NAME="runwayml/stable-diffusion-v1-5" diff --git a/docs/source/en/training/text2image.mdx b/docs/source/en/training/text2image.mdx index 70f8c003a787..dabb68397f78 100644 --- a/docs/source/en/training/text2image.mdx +++ b/docs/source/en/training/text2image.mdx @@ -72,7 +72,9 @@ To load a checkpoint to resume training, pass the argument `--resume_from_checkp -Launch the [PyTorch training script](https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/train_text_to_image.py) for a fine-tuning run on the [Pokémon BLIP captions](https://huggingface.co/datasets/lambdalabs/pokemon-blip-captions) dataset like this: +Launch the [PyTorch training script](https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/train_text_to_image.py) for a fine-tuning run on the [Pokémon BLIP captions](https://huggingface.co/datasets/lambdalabs/pokemon-blip-captions) dataset like this. + +Specify the `MODEL_NAME` environment variable (either a Hub model repository id or a path to the directory containing the model weights) and pass it to the [`~diffusers.DiffusionPipeline.from_pretrained.pretrained_model_name_or_path`] argument. {"path": "../../../../examples/text_to_image/README.md", @@ -141,6 +143,8 @@ Before running the script, make sure you have the requirements installed: pip install -U -r requirements_flax.txt ``` +Specify the `MODEL_NAME` environment variable (either a Hub model repository id or a path to the directory containing the model weights) and pass it to the [`~diffusers.DiffusionPipeline.from_pretrained.pretrained_model_name_or_path`] argument. + Now you can launch the [Flax training script](https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/train_text_to_image_flax.py) like this: ```bash diff --git a/docs/source/en/training/text_inversion.mdx b/docs/source/en/training/text_inversion.mdx index 6e6971d7f119..168400a7f28e 100644 --- a/docs/source/en/training/text_inversion.mdx +++ b/docs/source/en/training/text_inversion.mdx @@ -83,7 +83,9 @@ To resume training from a saved checkpoint, pass the following argument to the t For your training dataset, download these [images of a cat statue](https://drive.google.com/drive/folders/1fmJMs25nxS_rSNqS5hTcRdLem_YQXbq5) and store them in a directory. -Set the `MODEL_NAME` environment variable to the model repository id, and the `DATA_DIR` environment variable to the path of the directory containing the images. Now you can launch the [training script](https://github.com/huggingface/diffusers/blob/main/examples/textual_inversion/textual_inversion.py): +Specify the `MODEL_NAME` environment variable (either a Hub model repository id or a path to the directory containing the model weights) and pass it to the [`~diffusers.DiffusionPipeline.from_pretrained.pretrained_model_name_or_path`] argument, and the `DATA_DIR` environment variable to the path of the directory containing the images. + +Now you can launch the [training script](https://github.com/huggingface/diffusers/blob/main/examples/textual_inversion/textual_inversion.py): @@ -121,6 +123,8 @@ Before you begin, make sure you install the Flax specific dependencies: pip install -U -r requirements_flax.txt ``` +Specify the `MODEL_NAME` environment variable (either a Hub model repository id or a path to the directory containing the model weights) and pass it to the [`~diffusers.DiffusionPipeline.from_pretrained.pretrained_model_name_or_path`] argument. + Then you can launch the [training script](https://github.com/huggingface/diffusers/blob/main/examples/textual_inversion/textual_inversion_flax.py): ```bash From 6ffbab45536cae75d6371527b605a0daafe19de4 Mon Sep 17 00:00:00 2001 From: Steven Liu Date: Thu, 20 Apr 2023 13:10:36 -0400 Subject: [PATCH 2/2] apply feedback --- docs/source/en/training/dreambooth.mdx | 34 +++++++++++++++------- docs/source/en/training/text_inversion.mdx | 17 ++++++++--- 2 files changed, 37 insertions(+), 14 deletions(-) diff --git a/docs/source/en/training/dreambooth.mdx b/docs/source/en/training/dreambooth.mdx index 7be5b2a6a248..c5a5a047d114 100644 --- a/docs/source/en/training/dreambooth.mdx +++ b/docs/source/en/training/dreambooth.mdx @@ -50,6 +50,20 @@ from accelerate.utils import write_basic_config write_basic_config() ``` +Finally, download a [few images of a dog](https://huggingface.co/datasets/diffusers/dog-example) to DreamBooth with: + +```py +from huggingface_hub import snapshot_download + +local_dir = "./dog" +snapshot_download( + "diffusers/dog-example", + local_dir=local_dir, + repo_type="dataset", + ignore_patterns=".gitattributes", +) +``` + ## Finetuning @@ -60,13 +74,13 @@ DreamBooth finetuning is very sensitive to hyperparameters and easy to overfit. -Let's try DreamBooth with a [few images of a dog](https://drive.google.com/drive/folders/1BO_dyz-p65qhBRRMRA4TbZ8qW4rB99JZ); download and save them to a directory and then set the `INSTANCE_DIR` environment variable to that path. +Set the `INSTANCE_DIR` environment variable to the path of the directory containing the dog images. Specify the `MODEL_NAME` environment variable (either a Hub model repository id or a path to the directory containing the model weights) and pass it to the [`~diffusers.DiffusionPipeline.from_pretrained.pretrained_model_name_or_path`] argument. ```bash export MODEL_NAME="CompVis/stable-diffusion-v1-4" -export INSTANCE_DIR="path_to_training_images" +export INSTANCE_DIR="./dog" export OUTPUT_DIR="path_to_saved_model" ``` @@ -102,7 +116,7 @@ Now you can launch the training script with the following command: ```bash export MODEL_NAME="duongna/stable-diffusion-v1-4-flax" -export INSTANCE_DIR="path-to-instance-images" +export INSTANCE_DIR="./dog" export OUTPUT_DIR="path-to-save-model" python train_dreambooth_flax.py \ @@ -128,7 +142,7 @@ The authors recommend generating `num_epochs * num_samples` images for prior pre ```bash export MODEL_NAME="CompVis/stable-diffusion-v1-4" -export INSTANCE_DIR="path_to_training_images" +export INSTANCE_DIR="./dog" export CLASS_DIR="path_to_class_images" export OUTPUT_DIR="path_to_saved_model" @@ -153,7 +167,7 @@ accelerate launch train_dreambooth.py \ ```bash export MODEL_NAME="duongna/stable-diffusion-v1-4-flax" -export INSTANCE_DIR="path-to-instance-images" +export INSTANCE_DIR="./dog" export CLASS_DIR="path-to-class-images" export OUTPUT_DIR="path-to-save-model" @@ -190,7 +204,7 @@ Pass the `--train_text_encoder` argument to the training script to enable finetu ```bash export MODEL_NAME="CompVis/stable-diffusion-v1-4" -export INSTANCE_DIR="path_to_training_images" +export INSTANCE_DIR="./dog" export CLASS_DIR="path_to_class_images" export OUTPUT_DIR="path_to_saved_model" @@ -217,7 +231,7 @@ accelerate launch train_dreambooth.py \ ```bash export MODEL_NAME="duongna/stable-diffusion-v1-4-flax" -export INSTANCE_DIR="path-to-instance-images" +export INSTANCE_DIR="./dog" export CLASS_DIR="path-to-class-images" export OUTPUT_DIR="path-to-save-model" @@ -353,7 +367,7 @@ Then pass the `--use_8bit_adam` option to the training script: ```bash export MODEL_NAME="CompVis/stable-diffusion-v1-4" -export INSTANCE_DIR="path_to_training_images" +export INSTANCE_DIR="./dog" export CLASS_DIR="path_to_class_images" export OUTPUT_DIR="path_to_saved_model" @@ -382,7 +396,7 @@ To run DreamBooth on a 12GB GPU, you'll need to enable gradient checkpointing, t ```bash export MODEL_NAME="CompVis/stable-diffusion-v1-4" -export INSTANCE_DIR="path-to-instance-images" +export INSTANCE_DIR="./dog" export CLASS_DIR="path-to-class-images" export OUTPUT_DIR="path-to-save-model" @@ -429,7 +443,7 @@ Launch training with the following command: ```bash export MODEL_NAME="CompVis/stable-diffusion-v1-4" -export INSTANCE_DIR="path_to_training_images" +export INSTANCE_DIR="./dog" export CLASS_DIR="path_to_class_images" export OUTPUT_DIR="path_to_saved_model" diff --git a/docs/source/en/training/text_inversion.mdx b/docs/source/en/training/text_inversion.mdx index 168400a7f28e..e47a0519c704 100644 --- a/docs/source/en/training/text_inversion.mdx +++ b/docs/source/en/training/text_inversion.mdx @@ -1,4 +1,4 @@ -