[Request] Add Consistency models

[xenova]

Model/Pipeline/Scheduler description

OpenAI recently announced "Consistency" models - a new family of generative models that support fast one-step generation (without adversarial training).

According to the paper, "Consistency models are heavily inspired by the theory of continuous-time) diffusion models", so it may be fitting to include in this library... unless HF wants to create a new "consistency" library 🤣 (it might be too early to decide that however).

Open source status

• The model implementation is available
• The model weights are available (Only relevant if addition is not a scheduler).

Provide useful links for the implementation

Repo: https://github.com/openai/consistency_models
Paper: https://arxiv.org/abs/2303.01469
Pre-trained models: https://github.com/openai/consistency_models#pre-trained-models

[patrickvonplaten]

Wow lots of upvotes here! Is anybody in the community interested in picking this up? I'd love to guide someone!

[dg845]

I would be interested in trying my hand at this once I wrap up my current PR, if no one else has picked it up by then :).

[patrickvonplaten]

Also cc @yiyixuxu @williamberman in case you're interesting!

[tyshiwo1]

I made a simple implementation of consistency model (CT), but I do not have enough resources to verify if it performs as expected.

#3281

Also, this is my first time to use "pull requests", so I don't know what I should do next. Can anyone help me?

[dg845]

Hi, if you haven't taken a look yet, looking over the contributing guide, particularly the Adding pipelines, models, schedulers and How to write a good pull request (PR) sections, might be helpful.

I see you've closed your PR; if you re-open it, I think the maintainers will take a look and give some pointers and next steps (although it may take some time).

Hope this helps :).

[tyshiwo1]

Hi, if you haven't taken a look yet, looking over the contributing guide, particularly the Adding pipelines, models, schedulers and How to write a good pull request (PR) sections, might be helpful.

I see you've closed your PR; if you re-open it, I think the maintainers will take a look and give some pointers and next steps (although it may take some time).

Hope this helps :).

Thank you

[dg845]

Hi @tyshiwo1,

Are you planning to (re-)open a PR for a consistency models pipeline? If not, I can start working on one.

BTW, if you need computational resources to test your code, the maintainers will probably be able to help you :).

[patrickvonplaten]

@dg845, it'd be amazing if you could open a PR here! Happy to help in any way we can

[tyshiwo1]

Hi @tyshiwo1,

Are you planning to (re-)open a PR for a consistency models pipeline? If not, I can start working on one.

BTW, if you need computational resources to test your code, the maintainers will probably be able to help you :).

Oh, sorry, I just read it today. I have too much deadlines recently. I may not have time to re-open it. I would appreciate it if you open a new PR.

[ayushtues]

Hi @patrickvonplaten @dg845, I worked on adding Kandinksy recently to diffusers, with that almost wrapped up, I would love to start working on adding Consistency Models next, let me know if you want to work together on this @dg845!

[dg845]

@ayushtues Sounds good :).

I've opened up a PR with my current progress at #3492.

[patrickvonplaten]

Very cool! Let us know if you need help :-) Also cc @williamberman