[Dreambooth flux] bug fix for dreambooth script (align with dreambooth lora)

[linoytsaban]

fixes #9204 (comment)

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[sayakpaul]

Single comment.

[sayakpaul]

Where is this from?

diffusers/src/diffusers/pipelines/flux/pipeline_flux.py

Line 760 in 4e74206

latents = self._unpack_latents(latents, height, width, self.vae_scale_factor)

We don't additionally scale it by "/2".

[linoytsaban]

it's just a modification from the original version of the training scripts where we had

model_pred = FluxPipeline._unpack_latents(
                    model_pred,
                    height=int(model_input.shape[2] * 8),
                    width=int(model_input.shape[3] * 8),
                    vae_scale_factor=vae_scale_factor,
                )

and it didnt work with all resolutions, so we fixed in the previous PR for the LoRA script

(in the pipeline there is this- diffusers/src/diffusers/pipelines/flux/pipeline_flux.py)

[sayakpaul]

Yeah but we still don't have to do the additional scaling in the original pipeline, no? And it works with multiple resolutions without that. So, I am struggling to understand why we would need it here.

[linoytsaban]

in the pipeline we first scale in prepare_latents the width and height by

height = 2 * (int(height) // self.vae_scale_factor)
width = 2 * (int(width) // self.vae_scale_factor)

but we don't override them, so when they're sent to unpack_latents it's the x8 of the scaled version here^
in the training script, we send

height=model_input.shape[2],
width=model_input.shape[3],

to pack_latents - which is already a scaled down version because it happens after vae encoding.
i.e. model_input.shape[2] is equivalent to 2 * (int(height) // self.vae_scale_factor) in shape
that's why when we call unpack_latents in the training script we need to scale up

[sayakpaul]

Ah okay. Thanks for explaining this. Perhaps we could add a link to this comment in our script for our bookkeeping?

[linoytsaban]

done :)

[linoytsaban]

@sayakpaul shall we merge?

[sayakpaul]

Thank you!