Skip to content

Conversation

@dongyang0122
Copy link
Collaborator

Fixes #7858.

Integrating a tailored diffusion U-Net model into the generative model application to improve the production of high-dimensional 3D images, specifically sized at 512 x 512 x 512.

A few sentences describing the changes proposed in this pull request.

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • Breaking change (fix or new feature that would cause existing functionality to change).
  • New tests added to cover the changes.
  • Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
  • Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
  • In-line docstrings updated.
  • Documentation updated, tested make html command in the docs/ folder.

Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>
pre-commit-ci bot and others added 26 commits June 21, 2024 21:14
Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>
Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>
Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>
Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>
Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>
Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>
Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>
Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>
Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>
Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>
Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>
Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>
Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>
Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>
Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>
Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>
Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>
Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>
@dongyang0122 dongyang0122 marked this pull request as ready for review June 22, 2024 06:35
Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>
Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>
@dongyang0122
Copy link
Collaborator Author

@KumoLiu @Nic-Ma @ericspod I am not clear how to pass these failed checks. Please share your suggestion. Thanks.

Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>
Copy link
Contributor

@KumoLiu KumoLiu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR, few comments inline.

Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>
Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>
Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>
Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>
Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>
Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>
Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>
dongyang0122 and others added 6 commits July 1, 2024 13:13
Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>
Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>
Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>
Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>
Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>
@KumoLiu
Copy link
Contributor

KumoLiu commented Jul 1, 2024

/build

@KumoLiu KumoLiu requested review from ericspod and mingxin-zheng July 1, 2024 15:19
Copy link
Member

@ericspod ericspod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is good now. I'm not sure if referencing a specific commit in requirements-dev.txt is a good idea, we can do another "bug-fix" release of GenerativeModels later if this is a problem.

"generative.networks.nets.diffusion_model_unet", name="get_timestep_embedding"
)
get_up_block, has_get_up_block = optional_import("generative.networks.nets.diffusion_model_unet", name="get_up_block")
xformers, has_xformers = optional_import("xformers")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @dongyang0122 @KumoLiu , I think we need to either add xformers into our requirements-dev, or remove this from the code.

P.S. As far as I know, the installation of xformers is pretty nasty with nvidia DLFW pytorch container. It need to build from source and the compilation will somehow break/restart the machine if it hardwares are low profile.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Adding Tailored Model Implementations into Generative Model Application

4 participants