Sliding Patch WSI Dataset

[bhashemian]

Fixes #4179

Description

This PR implements SlidingPatchWSIDataset that extract patches from one or multiple whole slide images in a sliding-window manner. This is useful for the inference of the pathology pipelines.

Status

Ready

Types of changes

• Non-breaking change (fix or new feature that would not break existing functionality).
• New tests added to cover the changes.
• Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
• Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
• In-line docstrings updated.
• Documentation updated, tested make html command in the docs/ folder.

[bhashemian]

Hi @Nic-Ma can you too help to review this PR? Thanks

[wyli]

thanks, please see some comments inline

[wyli]

thanks, this enhances the sliding window functions and adds wsi sliding window support.

[wyli]

/build

[wyli]

there's a test error...

[2022-05-25T13:08:02.322Z] ======================================================================
[2022-05-25T13:08:02.322Z] FAIL: test_read_patches_9 (tests.test_sliding_patch_wsi_dataset.TestSlidingPatchWSIDatasetCuCIM)
[2022-05-25T13:08:02.322Z] ----------------------------------------------------------------------
[2022-05-25T13:08:02.322Z] Traceback (most recent call last):
[2022-05-25T13:08:02.322Z]   File "/opt/conda/lib/python3.8/site-packages/parameterized/parameterized.py", line 533, in standalone_func
[2022-05-25T13:08:02.322Z]     return func(*(a + p.args), **p.kwargs)
[2022-05-25T13:08:02.322Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_sliding_patch_wsi_dataset.py", line 227, in test_read_patches
[2022-05-25T13:08:02.322Z]     self.assertEqual(len(dataset), len(expected))
[2022-05-25T13:08:02.322Z] AssertionError: 1 != 2
[2022-05-25T13:08:02.322Z] 
[2022-05-25T13:08:02.322Z] ----------------------------------------------------------------------

with nvcr.io/nvidia/pytorch:22.04-py3 V100, please let me know if you need more info to debug

I think the class could inherit Randomizable to ensure the deterministic behaviour e.g.

MONAI/monai/apps/datasets.py

Line 33 in eb60b9c

class MedNISTDataset(Randomizable, CacheDataset):

[bhashemian]

there's a test error...

[2022-05-25T13:08:02.322Z] ======================================================================
[2022-05-25T13:08:02.322Z] FAIL: test_read_patches_9 (tests.test_sliding_patch_wsi_dataset.TestSlidingPatchWSIDatasetCuCIM)
[2022-05-25T13:08:02.322Z] ----------------------------------------------------------------------
[2022-05-25T13:08:02.322Z] Traceback (most recent call last):
[2022-05-25T13:08:02.322Z]   File "/opt/conda/lib/python3.8/site-packages/parameterized/parameterized.py", line 533, in standalone_func
[2022-05-25T13:08:02.322Z]     return func(*(a + p.args), **p.kwargs)
[2022-05-25T13:08:02.322Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_sliding_patch_wsi_dataset.py", line 227, in test_read_patches
[2022-05-25T13:08:02.322Z]     self.assertEqual(len(dataset), len(expected))
[2022-05-25T13:08:02.322Z] AssertionError: 1 != 2
[2022-05-25T13:08:02.322Z] 
[2022-05-25T13:08:02.322Z] ----------------------------------------------------------------------

with nvcr.io/nvidia/pytorch:22.04-py3 V100, please let me know if you need more info to debug

I think the class could inherit Randomizable to ensure the deterministic behaviour e.g.

MONAI/monai/apps/datasets.py

Line 33 in eb60b9c

class MedNISTDataset(Randomizable, CacheDataset):

@wyli
I'll add the Randomizable but with set_determinisim (and using numpy random), I'm wondering why it should produce different results! Do you have any idea?

[wyli]

I guess because there are multiple skipUnless in the tests, depending on the skipping conditions the np.random.randint is called various times:

MONAI/monai/data/wsi_datasets.py

Line 232 in b520d68

return tuple(np.random.randint(low, high) for low, high in offset_limits)

with the randomizable API, it's possible to use self.R.randint(low, high), so that it controls the local random state. pls see the seed usage: (

MONAI/tests/test_mednistdataset.py

Line 59 in db61a08

data = MedNISTDataset(root_dir=testing_dir, transform=transform, section="test", download=False, seed=42)

)

[wyli]

/build

[wyli]

/build

[wyli]

/build

[bhashemian]

@wyli, it seems that blossom-ci has failed. Can you provide details about it? Thanks

[wyli]

I think the current errors are irrelevant to the PR, I'm working on it