Fix potential type mismatch errors in SDXL pipelines

[Yukun-Huang]

What does this PR do?

Fixes #4752 potential type mismatch errors in SDXL pipelines.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline?
• Did you read our philosophy doc (important for complex PRs)?
• Was this discussed/approved via a Github issue or the forum? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sayakpaul

[sayakpaul]

@patrickvonplaten WDYT?

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[patrickvonplaten]

I still can't reproduce the bug sadly - could you add a reproduceable here @hyk1996 🙏

[Yukun-Huang]

I still can't reproduce the bug sadly - could you add a reproduceable here @hyk1996 🙏

@patrickvonplaten
I reproduce the issue on a Colab Notebook:
https://colab.research.google.com/gist/hyk1996/82ab6af1a9ad3d0b82b1f599f877c96a/scratchpad.ipynb
and it should be reproducible with torch < 2.0 and without xformers.

[patrickvonplaten]

Interesting! I can reproduce the problem

[patrickvonplaten]

Looking into it

[patrickvonplaten]

Hey @hyk1996,

I now understand what was meant - thanks a lot for bearing with us here!
The reason this bug was not caught is because almost everybody uses either xformers or PyTorch 2.0 when dealing with SDXL in which case the bug doesn't happen. Nevertheless this is a very important bug fix - great job!

I changed a couple of things in the PR to make sure everything now stays consistent between calls:

• The vae should actually not be touched when only the latents are returned
• The vae should be "back casted" to fp16 in it has been force upcasted before. This is important to ensure that the dtype not automagically changes which can be very confusing.

Are the changes as applied to your PR ok for you?

Also cc @sayakpaul for a final review maybe?

Thanks a bunch for fiinding this very important bug @hyk1996

[Yukun-Huang]

@patrickvonplaten
The changes are ok for me, glad to be of help :-)