Skip to content

Conversation

@nsrivathsa
Copy link
Contributor

Fixes # .

Description

Added a Transform for CuPy-based Stain Normalization for Pathology.

Status

Work in progress

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • Breaking change (fix or new feature that would cause existing functionality to change).
  • New tests added to cover the changes.
  • Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
  • Quick tests passed locally by running ./runtests.sh --quick --unittests.
  • In-line docstrings updated.
  • Documentation updated, tested make html command in the docs/ folder.

@nsrivathsa nsrivathsa changed the title [Draft] Adding Stain Normalization Transform for Pathology [WIP] Adding Stain Normalization Transform for Pathology Mar 31, 2021
@bhashemian bhashemian self-requested a review April 1, 2021 00:17
Copy link
Member

@bhashemian bhashemian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well done! It's a great job!
I left few comments but once you add the unittests, I will review it again. Thanks

Copy link
Member

@bhashemian bhashemian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left few comments on cupy. Please let me know if you have any question on that.

Copy link
Member

@bhashemian bhashemian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @nsrivathsa, thanks for addressing all the comments. So far so good! Have you had a chance to work on the test cases?

bhashemian and others added 25 commits April 12, 2021 15:42
Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
* Rename the prob map producer unittest to match the module

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

* Change probs_map to prob_map

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

* Prepend image_inference_outputs with temp to be ignored

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
Signed-off-by: Yiheng Wang <vennw@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
* followup of Project-MONAI#1878, fixes tests, remove json loading

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* Update test ordinal numbers

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

Co-authored-by: Behrooz <3968947+behxyz@users.noreply.github.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
* clip variance to be >= 0

Signed-off-by: kate-sann5100 <yiwen.li@st-annes.ox.ac.uk>

* max for torch1.6

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

Co-authored-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
* update to use pytorch2103

Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
…MONAI#1922)

* [DLMED] SaveImage supports patch data

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] fix flake8 issue

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] fix flake8 issue

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] fix flake8 issue

Signed-off-by: Nic Ma <nma@nvidia.com>

Co-authored-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
This reverts commit 4bd26f9.

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

Co-authored-by: Nic Ma <nma@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
* [DLMED] add EarlyStop handler

Signed-off-by: Nic Ma <nma@nvidia.com>

* [MONAI] python code formatting

Signed-off-by: monai-bot <monai.miccai2019@gmail.com>

* [DLMED] enhance validation handler

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] add set_trainer support

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] add more check

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] fix flake8 issue

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] update according to comments

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] fix flake8 issue

Signed-off-by: Nic Ma <nma@nvidia.com>

Co-authored-by: monai-bot <monai.miccai2019@gmail.com>
Co-authored-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
* [DLMED] change to save patch_index in meta_dict

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] remove error import

Signed-off-by: Nic Ma <nma@nvidia.com>

* [MONAI] python code formatting

Signed-off-by: monai-bot <monai.miccai2019@gmail.com>

* [DLMED] fix CI tests

Signed-off-by: Nic Ma <nma@nvidia.com>

Co-authored-by: monai-bot <monai.miccai2019@gmail.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
* Enabled partial checkpoint loading

Allowing partial loading of a model via strict=False.

Signed-off-by: Petru-Daniel Tudosiu <petru.daniel@tudosiu.com>

* [MONAI] python code formatting

Signed-off-by: monai-bot <monai.miccai2019@gmail.com>

* [DLMED] simplify strict arg

Signed-off-by: Nic Ma <nma@nvidia.com>

Co-authored-by: monai-bot <monai.miccai2019@gmail.com>
Co-authored-by: Nic Ma <nma@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
* less warning msg; remove PILImage types

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* remove engine type

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* temp tests

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* add quick py36 37 tests

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* temp tests

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* Revert "temp tests"

This reverts commit deaed40.

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* Revert "temp tests"

This reverts commit 3e8d8f4.

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* min test exclude senet test

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* temp test

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* Revert "temp test"

This reverts commit 31f40a0.

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update get gpu id

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update docsstrings

Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
* Add ProbNMS to transforts/post/array

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

* Implement ProbNMSDict

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

* Update the usage and add unittests

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

* Update docs

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

* Correct docs

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

* Correct test case

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

Co-authored-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
* Implement garbage collector handler

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

* Make trigger_event lower case

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

* Add unittest for garbage collector

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

* Update docs

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

* Exclude from min test

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

* Fix a typo

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

* Fix a bug

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
* [DLMED] add strict_shape option

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] add unit tests

Signed-off-by: Nic Ma <nma@nvidia.com>

* update test case

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes test config

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

Co-authored-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
wyli and others added 27 commits April 12, 2021 15:42
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
* refactor docker building

Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
* update docs deps.

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes https://github.com/Project-MONAI/MONAI/runs/2283148095\?check_suite_focus\=true\#step:7:8972

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* Revert "fixes https://github.com/Project-MONAI/MONAI/runs/2283148095\?check_suite_focus\=true\#step:7:8972"

This reverts commit 1407400.

Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
* Update garbage collection assertion to a more reliable one

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>

* Remove print

Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
* Working ParameterScheduler 

Added a new ParameterScheduler handler and the required tests.

Signed-off-by: Petru-Daniel Tudosiu <petru.daniel@tudosiu.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
Signed-off-by: Nic Ma <nma@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
* [DLMED] add support for addtional events

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] add unit tests

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] fix typehints

Signed-off-by: Nic Ma <nma@nvidia.com>

* [MONAI] python code formatting

Signed-off-by: monai-bot <monai.miccai2019@gmail.com>

* fixes typos

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

Co-authored-by: monai-bot <monai.miccai2019@gmail.com>
Co-authored-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
* fixes Project-MONAI#1965

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* adds docstring

Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
…#1969)

* [DLMED] enhance doc-strings

Signed-off-by: Nic Ma <nma@nvidia.com>

* [MONAI] python code formatting

Signed-off-by: monai-bot <monai.miccai2019@gmail.com>

Co-authored-by: monai-bot <monai.miccai2019@gmail.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
Signed-off-by: Alvin Ihsani <nsrivathsa@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
* adding init efficientnet support

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

* fixing flake8 and further refactoring

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

* adding unittests for efficiennet

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

* making unittests backwards compatible python<3.8

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

* fixed kitty unittests file path

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

* adding docstrings and minor refactoring

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

* fix flake8-py3 failing test

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

* generalize drop_connect for n-dim, fix/add unittests, remove assert

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

* fix failing unittest, CC0-license image for test

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

* refactoring code for review

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

* WIP fix mypy type hint errors

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

* fix cuda test error

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

* WIP fix test errors

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

* adding non-default shape tests

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

* remove 3d case from non-default shape test

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

* refactoring and updating docs

Signed-off-by: masadcv <muhammad.asad@kcl.ac.uk>

Co-authored-by: Yiheng Wang <68361391+yiheng-wang-nv@users.noreply.github.com>
Co-authored-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
* [DLMED] add TransformInverter handler

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] fix typo

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] add support in SegmentationSaver handler

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] fix flake8 issue

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] fix flake8 issue

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] fix CI test

Signed-off-by: Nic Ma <nma@nvidia.com>

* [MONAI] python code formatting

Signed-off-by: monai-bot <monai.miccai2019@gmail.com>

Co-authored-by: monai-bot <monai.miccai2019@gmail.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
* changelog for 0.5.0

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update changelog

Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
* [DLMED] add TransformInverter handler

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] fix typo

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] add support in SegmentationSaver handler

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] fix flake8 issue

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] fix flake8 issue

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] fix CI test

Signed-off-by: Nic Ma <nma@nvidia.com>

* [MONAI] python code formatting

Signed-off-by: monai-bot <monai.miccai2019@gmail.com>

* [DLMED] save mode into inverse dict

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] add unit tests

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] fix ToTensor inverse issue

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] change the replacement logic into util function

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] add more tests

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] fix flake8

Signed-off-by: Nic Ma <nma@nvidia.com>

* [MONAI] python code formatting

Signed-off-by: monai-bot <monai.miccai2019@gmail.com>

Co-authored-by: monai-bot <monai.miccai2019@gmail.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
* [DLMED] update highlights for 0.5

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] add invert transforms

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] add more highlights

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] add checkpointloader

Signed-off-by: Nic Ma <nma@nvidia.com>

* downscale image

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* downscale image

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update docs

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update desc

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

Co-authored-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
* update convs docstrings

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* increase tol for cuda crf

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update requirements according to docs/requirements

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* prune image

Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
Signed-off-by: Wenqi Li <wenqil@nvidia.com>

Co-authored-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
…NAI#1988)

* [DLMED] metrics support a list of tensor

Signed-off-by: Nic Ma <nma@nvidia.com>

* update workflow test

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* [DLMED] fix engine.state.output dict copy issue

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] add num_workers

Signed-off-by: Nic Ma <nma@nvidia.com>

* fixes typos

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update tests

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update docstrings

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* tests

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

Co-authored-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
* handling error

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* temp test

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes tests

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* Revert "temp test"

This reverts commit 60661ae.

Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
* update dev version

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* typo fix

Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
Signed-off-by: Richard Brown <33289025+rijobro@users.noreply.github.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
Signed-off-by: Behrooz <3968947+behxyz@users.noreply.github.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
Signed-off-by: Neha Srivathsa <nsrivathsa@nvidia.com>
@bhashemian
Copy link
Member

closing in favor of #1998

@bhashemian bhashemian closed this Apr 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants