Redesign whole slide image reading #4107

bhashemian · 2022-04-12T01:20:13Z

Description

This PR is a re-design and re-implementation of WSIReader while keeping the API the same as before and provide full backward compatibility (support for all the backends will come in the next PRs). This PR implements BaseWSIReader, WSIReader, and CuCIMWSIReader.

Types of changes

Non-breaking change (fix or new feature that would not break existing functionality).
Breaking change (fix or new feature that would cause existing functionality to change).
New tests added to cover the changes.
Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
In-line docstrings updated.
Documentation updated, tested make html command in the docs/ folder.

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

for more information, see https://pre-commit.ci

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

…ader

monai/data/wsi_reader.py

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

…sireader

monai/data/wsi_reader.py

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

bhashemian · 2022-04-18T15:02:07Z

Hi @Nic-Ma, do you have any other comments here? Thanks

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Nic-Ma

Hi @drbeh ,

Thanks for your quick update!
I put some minor comments inline, others look good to me.

Thanks.

tests/test_wsireader.py

monai/data/wsi_reader.py

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

…sireader

bhashemian · 2022-04-19T14:27:31Z

Hi @Nic-Ma, thanks for your comments. I have addressed them all and will merge this PR but please let me know if you have any comment and I will include it in the following PRs.

This reverts commit 78340a9.

Nic-Ma · 2022-04-19T14:46:27Z

Cool, thanks for your quick update.
Looks good to me.

* Redesign whole slide image reading (#4107) * Redesign BaseWSIReader, WSIReader, CuCIMWSIReader Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add unittests for WSIReader Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add image mode for output validation Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update docs Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update references to new WSIReader Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Remove legacy WSIReader Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update unittests Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update docs Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * sort imports Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Clean up imports Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update docstrings Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update docs and docstrings Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix a typo Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Remove redundant checking Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update read and other methods Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update wsireader to support multi image and update docstrings Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Make workaround for CuImage objects Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add unittests for multi image reading Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update a note about cucim Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update type hints and docstrings Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Implement Split transform Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add unittests Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update formatting Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Implement SplitDict Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add unittests for SplitDict Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add docs Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Remove images from docs Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Address all comments Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add example and size check Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update docs Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Revert references to new wsireader Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add missing comma Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* 4095 Add bundle download (#4114) * draft download Signed-off-by: Yiheng Wang <vennw@nvidia.com> * update bundle download Signed-off-by: Yiheng Wang <vennw@nvidia.com> * add url and load Signed-off-by: Yiheng Wang <vennw@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * rename args and remove a few places Signed-off-by: Yiheng Wang <vennw@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix flake8 issue Signed-off-by: Yiheng Wang <vennw@nvidia.com> * enhance with reviews Signed-off-by: Yiheng Wang <vennw@nvidia.com> * add instantiate for load Signed-off-by: Yiheng Wang <vennw@nvidia.com> * fix black error Signed-off-by: Yiheng Wang <vennw@nvidia.com> * add unittest Signed-off-by: Yiheng Wang <vennw@nvidia.com> * add load to docs Signed-off-by: Yiheng Wang <vennw@nvidia.com> * add skip Signed-off-by: Yiheng Wang <vennw@nvidia.com> * add schemaerror Signed-off-by: Yiheng Wang <vennw@nvidia.com> * fix partial places Signed-off-by: Yiheng Wang <vennw@nvidia.com> * download zip bundle Signed-off-by: Yiheng Wang <vennw@nvidia.com> * [DLMED] restore Exception for test Signed-off-by: Nic Ma <nma@nvidia.com> * update ts features Signed-off-by: Yiheng Wang <vennw@nvidia.com> * add config_files test case Signed-off-by: Yiheng Wang <vennw@nvidia.com> * enhance docstring example for args_file Signed-off-by: Yiheng Wang <vennw@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Nic Ma <nma@nvidia.com> * Disable pylint error and fix CI tests of new tifffile (#4162) * workaround Signed-off-by: Nic Ma <nma@nvidia.com> * [DLMED] fix tifffile issue Signed-off-by: Nic Ma <nma@nvidia.com> * Fixed an error in DiNTS model implementation and enabled act and norm layer options (#4157) * fixed a bug Signed-off-by: dongy <dongy@nvidia.com> * autofix Signed-off-by: dongy <dongy@nvidia.com> * update test case Signed-off-by: dongy <dongy@nvidia.com> Co-authored-by: dongy <dongy@nvidia.com> * Split transform (#4153) * Redesign whole slide image reading (#4107) * Redesign BaseWSIReader, WSIReader, CuCIMWSIReader Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add unittests for WSIReader Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add image mode for output validation Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update docs Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update references to new WSIReader Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Remove legacy WSIReader Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update unittests Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update docs Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * sort imports Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Clean up imports Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update docstrings Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update docs and docstrings Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix a typo Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Remove redundant checking Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update read and other methods Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update wsireader to support multi image and update docstrings Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Make workaround for CuImage objects Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add unittests for multi image reading Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update a note about cucim Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update type hints and docstrings Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Implement Split transform Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add unittests Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update formatting Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Implement SplitDict Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add unittests for SplitDict Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add docs Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Remove images from docs Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Address all comments Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add example and size check Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update docs Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Revert references to new wsireader Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add missing comma Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * fix bundle download test issue (#4169) Signed-off-by: Yiheng Wang <vennw@nvidia.com> * 4094 Enhance `ckpt_export` to save config files (#4159) * [DLMED] enhance checkpoint export Signed-off-by: Nic Ma <nma@nvidia.com> * [DLMED] update according to comments Signed-off-by: Nic Ma <nma@nvidia.com> * Move RGB/RGBA checks to base class (#4171) Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> Co-authored-by: Nic Ma <nma@nvidia.com> * [CICD] To support temp dgx runner (#4175) * Support new temp dgx runner Signed-off-by: Peixin Li <pxli@nyu.edu> * atol 1e-5 Signed-off-by: Wenqi Li <wenqil@nvidia.com> Co-authored-by: Wenqi Li <wenqil@nvidia.com> * Test fix for AMP kwargs (#4178) Signed-off-by: Eric Kerfoot <eric.kerfoot@kcl.ac.uk> Co-authored-by: Yiheng Wang <68361391+yiheng-wang-nv@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Nic Ma <nma@nvidia.com> Co-authored-by: dongyang0122 <don.yang.mech@gmail.com> Co-authored-by: dongy <dongy@nvidia.com> Co-authored-by: Behrooz <3968947+drbeh@users.noreply.github.com> Co-authored-by: Peixin <pxli@nyu.edu> Co-authored-by: Eric Kerfoot <17726042+ericspod@users.noreply.github.com>

* Redesign whole slide image reading (Project-MONAI#4107) * Redesign BaseWSIReader, WSIReader, CuCIMWSIReader Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add unittests for WSIReader Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add image mode for output validation Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update docs Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update references to new WSIReader Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Remove legacy WSIReader Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update unittests Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update docs Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * sort imports Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Clean up imports Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update docstrings Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update docs and docstrings Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix a typo Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Remove redundant checking Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update read and other methods Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update wsireader to support multi image and update docstrings Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Make workaround for CuImage objects Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add unittests for multi image reading Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update a note about cucim Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update type hints and docstrings Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Implement Split transform Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add unittests Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update formatting Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Implement SplitDict Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add unittests for SplitDict Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add docs Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Remove images from docs Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Address all comments Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add example and size check Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update docs Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Revert references to new wsireader Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add missing comma Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

bhashemian added 4 commits April 12, 2022 01:18

Redesign BaseWSIReader, WSIReader, CuCIMWSIReader

2baa344

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Add unittests for WSIReader

e46fea0

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Add image mode for output validation

f946a63

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Merge branch 'dev' into new-wsireader

9b41019

bhashemian changed the base branch from dev to pathology April 12, 2022 20:44

bhashemian added 8 commits April 13, 2022 13:08

Update docs

9c15ea3

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Update references to new WSIReader

e004538

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Remove legacy WSIReader

df0a61e

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Update unittests

b64c087

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Update docs

7f09e73

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

sort imports

d95e1c2

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Clean up imports

025376b

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Update docstrings

3e6fb27

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

bhashemian marked this pull request as ready for review April 13, 2022 20:32

bhashemian requested review from Nic-Ma and wyli April 13, 2022 20:33

bhashemian and others added 5 commits April 13, 2022 20:36

Update docs and docstrings

b8a2444

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

0268a03

for more information, see https://pre-commit.ci

Fix a typo

f2b14aa

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Remove redundant checking

aed0ac5

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Merge branch 'new-wsireader' of github.com:drbeh/MONAI into new-wsire…

272d11f

…ader

Nic-Ma reviewed Apr 14, 2022

View reviewed changes

monai/data/wsi_reader.py Outdated Show resolved Hide resolved

monai/data/wsi_reader.py Show resolved Hide resolved

bhashemian added 2 commits April 14, 2022 17:34

Update read and other methods

ae4ff17

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Merge branch 'pathology' of github.com:Project-MONAI/MONAI into new-w…

e0bad98

…sireader

Nic-Ma reviewed Apr 15, 2022

View reviewed changes

monai/data/wsi_reader.py Outdated Show resolved Hide resolved

monai/data/wsi_reader.py Show resolved Hide resolved

monai/data/wsi_reader.py Show resolved Hide resolved

bhashemian added 3 commits April 18, 2022 14:43

Update wsireader to support multi image and update docstrings

e5a7a18

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Make workaround for CuImage objects

da6b675

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Add unittests for multi image reading

1ed8154

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Update a note about cucim

77f8747

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Nic-Ma reviewed Apr 19, 2022

View reviewed changes

bhashemian added 2 commits April 19, 2022 13:52

Update type hints and docstrings

36e5b17

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Merge branch 'pathology' of github.com:Project-MONAI/MONAI into new-w…

0328fe6

…sireader

bhashemian merged commit 78340a9 into Project-MONAI:pathology Apr 19, 2022

bhashemian added a commit that referenced this pull request Apr 19, 2022

Revert "Redesign whole slide image reading (#4107)"

36c77bb

This reverts commit 78340a9.

bhashemian mentioned this pull request Apr 19, 2022

New wsireader #4147

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Redesign whole slide image reading #4107

Redesign whole slide image reading #4107

Uh oh!

bhashemian commented Apr 12, 2022 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bhashemian commented Apr 18, 2022

Uh oh!

Nic-Ma left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bhashemian commented Apr 19, 2022

Uh oh!

Nic-Ma commented Apr 19, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Redesign whole slide image reading #4107

Redesign whole slide image reading #4107

Uh oh!

Conversation

bhashemian commented Apr 12, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bhashemian commented Apr 18, 2022

Uh oh!

Nic-Ma left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bhashemian commented Apr 19, 2022

Uh oh!

Nic-Ma commented Apr 19, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

bhashemian commented Apr 12, 2022 •

edited

Loading