controlnet inpaint with multiple conditions

[timegate]

I tried to use multiple conditions on controlnet inpaint (https://github.com/timegate/diffusers, aab4781),

but it (seg + depth) showed worse results than just one condition (either seg or depth)
Is there anybody working on this?

[timegate]

I succeed to get reasonable results by giving full-region conditions to depth map and seg map. But it is still interesting that one condition works better when using the masked version of the depth map or seg map.

[alexblattner]

@timegate do you happen to know how to make multicontrolnet work with paint with words? I'd like my openpose characters to be the correct ones and not have them switch around. Your work is really cool btw

[patrickvonplaten]

Thanks for the PR @timegate - I think it's time to move controlnet img2img and inpaint into native src/diffusers

[timegate]

@timegate do you happen to know how to make multicontrolnet work with paint with words? I'd like my openpose characters to be the correct ones and not have them switch around. Your work is really cool btw

I'm sorry but I can't be sure as I haven't looked at the code (paint with words), but I'm guessing it'll probably work in a similar context to this commit. I'll take a look if I have some time to do it. Thanks so much.

[timegate]

I close this issue as the pull request is done, and I will open another issue again if there are any additional updates / issues. Thanks for reviewing again :)