Bug fix and improvement in WSI #4216

bhashemian · 2022-05-02T15:34:23Z

Description

Fixes a bug and makes some updates in WSIReader and PatchWSIDataset

Status

Ready

Types of changes

Non-breaking change (fix or new feature that would not break existing functionality).

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

…rtcache

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

…rtcache

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

bhashemian · 2022-05-02T16:56:07Z

@Nic-Ma, I removed SmartCacheWSIDataset from this PR and kept only the minor improvement and bug fix. Can you please review it?

Nic-Ma · 2022-05-03T09:09:12Z

Hi @drbeh ,

I think we don't have missing action items for moving the existing WSI components to core, could you please help also unify test_wsireader_new.py and test_wsireader.py in this PR?
Please feel free to correct me if I misunderstood anything.

Thanks in advance.

monai/data/wsi_reader.py

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

bhashemian · 2022-05-03T14:05:25Z

Hi @drbeh ,

I think we don't have missing action items for moving the existing WSI components to core, could you please help also unify test_wsireader_new.py and test_wsireader.py in this PR? Please feel free to correct me if I misunderstood anything.

Thanks in advance.

Hi @Nic-Ma, I don't think that we should combine them in a single file since they are two different components with similar names. What I am planing to do is following:

First, update the pipelines and tutorials with the new components and make sure that they are working as expected.
Next, deprecate the old components.
Finally, rename test_xxx.py -> test_xxx_depricated.py and test_xxx_new.py -> test_xxx.py.

Please let me know if you think differently.

Nic-Ma · 2022-05-03T14:41:19Z

Hi @drbeh ,

I feel we may not need to keep the deprecated APIs, delete the APIs and tests directly?
@wyli What do you think?

Thanks.

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

bhashemian · 2022-05-03T14:53:32Z

Hi @drbeh ,

I feel we may not need to keep the deprecated APIs, delete the APIs and tests directly? @wyli What do you think?

Thanks.

Deleting a component directly is not a good practice. The way that is usually done is to deprecate the component first to notify users (via the deprecation message) and give them time to update their code. Then after a period of time delete them all.

Nic-Ma · 2022-05-03T15:19:09Z

Hi @drbeh ,

I understand the deprecate process, we already have several deprecated modules, for example:
https://github.com/Project-MONAI/MONAI/blob/dev/monai/data/png_writer.py#L22
deprecate may cause the code hard to read, so sometimes we just make breaking changes directly, depends how hard users switch to the new APIs.
Anyway, it's not a blocker issue, if @wyli @ericspod you guys also like the deprecate way, I am OK with it.

Thanks in advance.

bhashemian · 2022-05-04T14:16:59Z

Hi @Nic-Ma, I'll create an issue to continue the discussion about deprecating the pathology components or not. But do you have any other comment for this PR? thanks

Nic-Ma · 2022-05-04T14:34:53Z

Hi @drbeh ,

We are still in Labor's Day holiday today, I will try to review it again later.
You can ask @wyli @ericspod to review it first.

Thanks.

bhashemian · 2022-05-04T14:36:28Z

Sure, @Nic-Ma, enjoy your holiday! :)

Nic-Ma

Looks good to me.

Thanks.

Nic-Ma · 2022-05-05T00:25:07Z

/build

wyli · 2022-05-05T04:14:14Z

the PR description says it fixes a bug, was wondering if the relevant test cases should be updated to cover the change?

bhashemian · 2022-05-06T13:55:43Z

the PR description says it fixes a bug, was wondering if the relevant test cases should be updated to cover the change?

That's a great point @wyli! I'll add the test.

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

wyli · 2022-05-06T17:43:34Z

/build

* Make all transforms optional Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update wsireader tests Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Remove optional from PersistentDataset and its derivatives Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add unittests for cache without transform Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add default replace_rate Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add default value Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Set default replace_rate to 0.1 Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update metadata to include path Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Adds SmartCachePatchWSIDataset Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add unittests for SmartCachePatchWSIDataset Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update references Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update docs Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Remove smart cache Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Remove unused imports Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add path metadata for OpenSlide Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update metadata to be unified across different backends Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update wsi metadata for multi wsi objects Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add unittests for wsi metadata Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* add swin_unetr model (#4074) * add swin_unetr model Signed-off-by: kbressem <kenobressem@gmail.com> * 4217 Update PyTorch docker to 22.04 (#4218) * [DLMED] update to 22.04 Signed-off-by: Nic Ma <nma@nvidia.com> * fixes unit test tests.test_lr_finder Signed-off-by: Wenqi Li <wenqil@nvidia.com> * test new_empty Signed-off-by: Wenqi Li <wenqil@nvidia.com> Co-authored-by: Wenqi Li <wenqil@nvidia.com> Co-authored-by: Wenqi Li <831580+wyli@users.noreply.github.com> Signed-off-by: kbressem <kenobressem@gmail.com> * Add InstanceNorm3dNVFuser support (#4194) * implement the base class Signed-off-by: Yiheng Wang <vennw@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add unittest Signed-off-by: Yiheng Wang <vennw@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * autofix Signed-off-by: Yiheng Wang <vennw@nvidia.com> * switch to call apex directly Signed-off-by: Yiheng Wang <vennw@nvidia.com> * uncomment unittest Signed-off-by: Yiheng Wang <vennw@nvidia.com> * add apex install link in docstring Signed-off-by: Yiheng Wang <vennw@nvidia.com> * add channels_last_3d test case Signed-off-by: Yiheng Wang <vennw@nvidia.com> * rewrite types Signed-off-by: Yiheng Wang <vennw@nvidia.com> * change types Signed-off-by: Yiheng Wang <vennw@nvidia.com> * add docstrings Signed-off-by: Yiheng Wang <vennw@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Wenqi Li <831580+wyli@users.noreply.github.com> Signed-off-by: kbressem <kenobressem@gmail.com> * Update dice.py (#4234) * Update dice.py reduce redundant operations in DiceFocalLoss, initially caused oom Signed-off-by: Ryan Clanton <55164720+ryancinsight@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Ryan Clanton <55164720+ryancinsight@users.noreply.github.com> * [MONAI] python code formatting Signed-off-by: monai-bot <monai.miccai2019@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: monai-bot <monai.miccai2019@gmail.com> Signed-off-by: kbressem <kenobressem@gmail.com> * Bug fix and improvement in WSI (#4216) * Make all transforms optional Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update wsireader tests Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Remove optional from PersistentDataset and its derivatives Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add unittests for cache without transform Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add default replace_rate Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add default value Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Set default replace_rate to 0.1 Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update metadata to include path Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Adds SmartCachePatchWSIDataset Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add unittests for SmartCachePatchWSIDataset Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update references Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update docs Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Remove smart cache Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Remove unused imports Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add path metadata for OpenSlide Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update metadata to be unified across different backends Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Update wsi metadata for multi wsi objects Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> * Add unittests for wsi metadata Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com> Signed-off-by: kbressem <kenobressem@gmail.com> * Replace module (#4245) * replace modules Signed-off-by: Richard Brown <33289025+rijobro@users.noreply.github.com> * fix Signed-off-by: Richard Brown <33289025+rijobro@users.noreply.github.com> * replace_module -> replace_modules Signed-off-by: Richard Brown <33289025+rijobro@users.noreply.github.com> * fix Signed-off-by: Richard Brown <33289025+rijobro@users.noreply.github.com> Signed-off-by: kbressem <kenobressem@gmail.com> * Add GaussianSmooth as antialiasing filter in Resize (#4249) Signed-off-by: Can Zhao <canz@nvidia.com> Signed-off-by: kbressem <kenobressem@gmail.com> * 4235 fix 2204 nvfuser issue (#4241) * reproduce issue Signed-off-by: Yiheng Wang <vennw@nvidia.com> * remove 22.01 02 Signed-off-by: Yiheng Wang <vennw@nvidia.com> * remove other workflows Signed-off-by: Yiheng Wang <vennw@nvidia.com> * run on pull request Signed-off-by: Yiheng Wang <vennw@nvidia.com> * remove sleep Signed-off-by: Yiheng Wang <vennw@nvidia.com> * test single layer forward Signed-off-by: Yiheng Wang <vennw@nvidia.com> * add has_nvfuser Signed-off-by: Yiheng Wang <vennw@nvidia.com> * add check within factory Signed-off-by: Yiheng Wang <vennw@nvidia.com> * revert to original cron.yml Signed-off-by: Yiheng Wang <vennw@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix old pt issue Signed-off-by: Yiheng Wang <vennw@nvidia.com> * change to return directly if no cuda Signed-off-by: Yiheng Wang <vennw@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: kbressem <kenobressem@gmail.com> * Update to Bundle Specifiation (#4250) * Update to bundle specifiation Signed-off-by: Eric Kerfoot <eric.kerfoot@kcl.ac.uk> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding description in spec discussing the saved Torchscript object's file storage behaviour, and tweaking ckpt_export to add .json extension Signed-off-by: Eric Kerfoot <eric.kerfoot@kcl.ac.uk> * Annotating optional bundle files Signed-off-by: Eric Kerfoot <eric.kerfoot@kcl.ac.uk> * Adjusted ckpt_export test Signed-off-by: Eric Kerfoot <eric.kerfoot@kcl.ac.uk> * Fix Signed-off-by: Eric Kerfoot <eric.kerfoot@kcl.ac.uk> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: kbressem <kenobressem@gmail.com> * Implement NrrdReader and NrrdImage classes Signed-off-by: kbressem <kenobressem@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: kbressem <kenobressem@gmail.com> * run auto style fixes on image_reader.py Signed-off-by: kbressem <kenobressem@gmail.com> * add NrrdReader to monai/data/__init__.py Signed-off-by: kbressem <kenobressem@gmail.com> * Change the way spatial information is handled in NrrdReader Signed-off-by: kbressem <kenobressem@gmail.com> * add tests for NrrdReader Signed-off-by: kbressem <kenobressem@gmail.com> * Add NrrdReader to list of possible readers for LoadImage Signed-off-by: kbressem <kenobressem@gmail.com> * autofix formating Signed-off-by: kbressem <kenobressem@gmail.com> * autofix formating Signed-off-by: kbressem <kenobressem@gmail.com> * change NrrdImage class to namedtuple and make flake8 happy Signed-off-by: kbressem <kenobressem@gmail.com> * Add pynrrd to requirements Signed-off-by: kbressem <kenobressem@gmail.com> * correct typing for namedtumple make flake8 happy Signed-off-by: kbressem <kenobressem@gmail.com> * Add pynrrd info to `get_optional_config_values` Changed NrrdImage to dataclass Signed-off-by: kbressem <kenobressem@gmail.com> * exclude test_nrrd_reader.py from min tests Signed-off-by: kbressem <kenobressem@gmail.com> * add pynrrd to config files Signed-off-by: kbressem <kenobressem@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Change the way space is handled in the header. Now, if space is not in header, it is assumed to be LPS and converted to RAS. If space is defined and not LPS, nothing is done to prevent wrong conversions. Signed-off-by: kbressem <kenobressem@gmail.com> * add `TestLoadSaveNrrd` where it is tested if a nrrd file, created by ITKWriter can be loaded again. 2D and 3D files with no channels are tested Signed-off-by: kbressem <kenobressem@gmail.com> * autofix format Co-authored-by: kbressem <kenobressem@charite.de>

bhashemian and others added 19 commits April 27, 2022 18:12

Make all transforms optional

f6716e2

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Merge branch 'dev' into fix-smartcache

6506edd

Merge branch 'dev' into fix-smartcache

ee59746

Merge branch 'dev' of github.com:Project-MONAI/MONAI into fix-smartcache

aa08be9

Update wsireader tests

f21fe5a

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Remove optional from PersistentDataset and its derivatives

1610bbc

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Add unittests for cache without transform

3d9516d

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Merge branch 'fix-smartcache' of github.com:behxyz/MONAI into fix-sma…

b1656f8

…rtcache

Add default replace_rate

a9f24c5

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Add default value

0d6450b

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Set default replace_rate to 0.1

20c4882

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Merge branch 'dev' into fix-smartcache

7096ad8

Update metadata to include path

f586386

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Adds SmartCachePatchWSIDataset

9dc6ca4

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Add unittests for SmartCachePatchWSIDataset

6ad1b30

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Update references

c548750

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Merge branch 'fix-smartcache' of github.com:behxyz/MONAI into fix-sma…

e52d846

…rtcache

Merge branch 'dev' of github.com:Project-MONAI/MONAI into fix-smartcache

bee7f87

Update docs

c874f8e

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

bhashemian requested a review from Nic-Ma May 2, 2022 15:35

Remove smart cache

21e08c4

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

bhashemian changed the title ~~Implement SmartCache for PatchWSIDataset~~ Bug fix and doc update WSI May 2, 2022

bhashemian changed the title ~~Bug fix and doc update WSI~~ Bug fix and improvement and doc update WSI May 2, 2022

Remove unused imports

a9de037

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

bhashemian enabled auto-merge (squash) May 2, 2022 16:58

bhashemian requested a review from wyli May 2, 2022 19:01

bhashemian changed the title ~~Bug fix and improvement and doc update WSI~~ Bug fix and improvement in WSI May 2, 2022

Nic-Ma reviewed May 3, 2022

View reviewed changes

monai/data/wsi_reader.py Outdated Show resolved Hide resolved

Add path metadata for OpenSlide

0e88792

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Update metadata to be unified across different backends

ffd439e

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Merge branch 'dev' into fix-smartcache

408519a

bhashemian mentioned this pull request May 4, 2022

Deprecating Old Pathology Components #4226

Closed

Nic-Ma approved these changes May 5, 2022

View reviewed changes

wyli disabled auto-merge May 5, 2022 04:20

bhashemian added 3 commits May 6, 2022 14:23

Merge branch 'dev' of github.com:Project-MONAI/MONAI into fix-smartcache

d4a8f55

Update wsi metadata for multi wsi objects

a7d2dcb

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Add unittests for wsi metadata

960c117

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

wyli enabled auto-merge (squash) May 6, 2022 17:44

bhashemian added this to the Blend pathology components into core and separate them in a submodule [P0] milestone May 6, 2022

wyli merged commit 209db9e into Project-MONAI:dev May 6, 2022

bhashemian deleted the fix-smartcache branch May 6, 2022 19:46

bhashemian self-assigned this Jul 11, 2022

Bug fix and improvement in WSI #4216

Bug fix and improvement in WSI #4216

Uh oh!

Conversation

bhashemian commented May 2, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Status

Types of changes

Uh oh!

bhashemian commented May 2, 2022

Uh oh!

Nic-Ma commented May 3, 2022

Uh oh!

Uh oh!

bhashemian commented May 3, 2022

Uh oh!

Nic-Ma commented May 3, 2022

Uh oh!

bhashemian commented May 3, 2022

Uh oh!

Nic-Ma commented May 3, 2022

Uh oh!

bhashemian commented May 4, 2022

Uh oh!

Nic-Ma commented May 4, 2022

Uh oh!

bhashemian commented May 4, 2022

Uh oh!

Nic-Ma left a comment

Choose a reason for hiding this comment

Uh oh!

Nic-Ma commented May 5, 2022

Uh oh!

wyli commented May 5, 2022

Uh oh!

bhashemian commented May 6, 2022

Uh oh!

wyli commented May 6, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

bhashemian commented May 2, 2022 •

edited

Loading