Skip to content

Conversation

@bhashemian
Copy link
Member

@bhashemian bhashemian commented Apr 20, 2022

Fixes #4013

Description

Implement Split transform that splits an image based on a grid. This transform is particularly useful in pathology pipelines.

Status

Ready/Work in progress/Hold

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • New tests added to cover the changes.
  • Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
  • Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
  • In-line docstrings updated.
  • Documentation updated, tested make html command in the docs/ folder.

* Redesign BaseWSIReader,  WSIReader, CuCIMWSIReader

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Add unittests for WSIReader

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Add image mode for output validation

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update docs

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update references to new WSIReader

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Remove legacy WSIReader

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update unittests

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update docs

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* sort imports

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Clean up imports

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update docstrings

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update docs and docstrings

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix a typo

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Remove redundant checking

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update read and other methods

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update wsireader to support multi image and update docstrings

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Make workaround for CuImage objects

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Add unittests for multi image reading

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update a note about cucim

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update type hints and docstrings

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>
Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>
Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>
Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>
Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>
Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>
Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>
@bhashemian bhashemian marked this pull request as ready for review April 20, 2022 19:05
@bhashemian bhashemian requested review from Nic-Ma and wyli April 20, 2022 19:06
Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>
Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>
Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>
Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>
Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>
@bhashemian
Copy link
Member Author

Any other comments, @Nic-Ma? Thanks

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>
Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>
@bhashemian bhashemian enabled auto-merge (squash) April 22, 2022 16:16
@bhashemian bhashemian requested a review from Nic-Ma April 22, 2022 16:22
Copy link
Contributor

@Nic-Ma Nic-Ma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick update!
Looks good to me.
Let's address the sample image issue in another ticket #4161 later.

Thanks.

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>
@bhashemian
Copy link
Member Author

Thank you @Nic-Ma! It just need a /build!

@Nic-Ma
Copy link
Contributor

Nic-Ma commented Apr 23, 2022

/build

1 similar comment
@Nic-Ma
Copy link
Contributor

Nic-Ma commented Apr 23, 2022

/build

@Nic-Ma
Copy link
Contributor

Nic-Ma commented Apr 23, 2022

Hi @drbeh ,

The Blossom CI failed due to these cuCIM / tifffile errors:
https://pipelines.actions.githubusercontent.com/serviceHosts/b9c0ace7-8939-446b-80c4-ee7a711ae9fe/_apis/pipelines/1/runs/45363/signedlogcontent/2?urlExpires=2022-04-23T02%3A55%3A51.0143983Z&urlSigningMethod=HMACV1&urlSignature=xQZGiCa2Q7diUmOHn8359OQby9lS7KkNJZL1HYq9oSI%3D

Could you please help take a look?

Thanks.

2022-04-23T02:27:55.1788893Z [2022-04-23T02:27:07.931Z] ======================================================================
2022-04-23T02:27:55.1789153Z [2022-04-23T02:27:07.931Z] ERROR: test_read_malformats_0 (tests.test_wsireader_new.TestCuCIM)
2022-04-23T02:27:55.1789423Z [2022-04-23T02:27:07.931Z] ----------------------------------------------------------------------
2022-04-23T02:27:55.1789624Z [2022-04-23T02:27:07.931Z] Traceback (most recent call last):
2022-04-23T02:27:55.1789973Z [2022-04-23T02:27:07.931Z]   File "/opt/conda/lib/python3.8/site-packages/parameterized/parameterized.py", line 533, in standalone_func
2022-04-23T02:27:55.1790175Z [2022-04-23T02:27:07.931Z]     return func(*(a + p.args), **p.kwargs)
2022-04-23T02:27:55.1790532Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_wsireader_new.py", line 187, in test_read_malformats
2022-04-23T02:27:55.1790721Z [2022-04-23T02:27:07.931Z]     file_path = save_gray_tiff(
2022-04-23T02:27:55.1791068Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_wsireader_new.py", line 106, in save_gray_tiff
2022-04-23T02:27:55.1791406Z [2022-04-23T02:27:07.931Z]     imwrite(filename, img_gray, shape=img_gray.shape, photometric="rgb")
2022-04-23T02:27:55.1791713Z [2022-04-23T02:27:07.931Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 989, in imwrite
2022-04-23T02:27:55.1791966Z [2022-04-23T02:27:07.931Z]     result = tif.write(data, shape=shape, dtype=dtype, **kwargs)
2022-04-23T02:27:55.1792273Z [2022-04-23T02:27:07.931Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 2133, in write
2022-04-23T02:27:55.1792451Z [2022-04-23T02:27:07.931Z]     raise RuntimeError(
2022-04-23T02:27:55.1792831Z [2022-04-23T02:27:07.931Z] RuntimeError: None != 2 <StoredShape(frames=1, separate_samples=1, depth=1, length=16, width=16, contig_samples=1, extrasamples=0)>
2022-04-23T02:27:55.1792963Z [2022-04-23T02:27:07.931Z] 
2022-04-23T02:27:55.1793186Z [2022-04-23T02:27:07.931Z] ======================================================================
2022-04-23T02:27:55.1793486Z [2022-04-23T02:27:07.931Z] ERROR: test_read_patches_cucim_0 (tests.test_lesion_froc.TestEvaluateTumorFROC)
2022-04-23T02:27:55.1793745Z [2022-04-23T02:27:07.931Z] ----------------------------------------------------------------------
2022-04-23T02:27:55.1793943Z [2022-04-23T02:27:07.931Z] Traceback (most recent call last):
2022-04-23T02:27:55.1794273Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 307, in setUp
2022-04-23T02:27:55.1794503Z [2022-04-23T02:27:07.931Z]     prepare_test_data()
2022-04-23T02:27:55.1794847Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 52, in prepare_test_data
2022-04-23T02:27:55.1795077Z [2022-04-23T02:27:07.931Z]     save_as_tif("temp_ground_truth_0", ground_truth)
2022-04-23T02:27:55.1795417Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 34, in save_as_tif
2022-04-23T02:27:55.1795663Z [2022-04-23T02:27:07.931Z]     imwrite(file_path, array, compress="jpeg", tile=(16, 16))
2022-04-23T02:27:55.1795963Z [2022-04-23T02:27:07.931Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 989, in imwrite
2022-04-23T02:27:55.1796211Z [2022-04-23T02:27:07.931Z]     result = tif.write(data, shape=shape, dtype=dtype, **kwargs)
2022-04-23T02:27:55.1796517Z [2022-04-23T02:27:07.931Z] TypeError: write() got an unexpected keyword argument 'compress'
2022-04-23T02:27:55.1796653Z [2022-04-23T02:27:07.931Z] 
2022-04-23T02:27:55.1796874Z [2022-04-23T02:27:07.931Z] ======================================================================
2022-04-23T02:27:55.1797161Z [2022-04-23T02:27:07.931Z] ERROR: test_read_patches_cucim_1 (tests.test_lesion_froc.TestEvaluateTumorFROC)
2022-04-23T02:27:55.1797428Z [2022-04-23T02:27:07.931Z] ----------------------------------------------------------------------
2022-04-23T02:27:55.1797627Z [2022-04-23T02:27:07.931Z] Traceback (most recent call last):
2022-04-23T02:27:55.1797955Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 307, in setUp
2022-04-23T02:27:55.1798129Z [2022-04-23T02:27:07.931Z]     prepare_test_data()
2022-04-23T02:27:55.1798472Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 52, in prepare_test_data
2022-04-23T02:27:55.1798702Z [2022-04-23T02:27:07.931Z]     save_as_tif("temp_ground_truth_0", ground_truth)
2022-04-23T02:27:55.1799037Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 34, in save_as_tif
2022-04-23T02:27:55.1799282Z [2022-04-23T02:27:07.931Z]     imwrite(file_path, array, compress="jpeg", tile=(16, 16))
2022-04-23T02:27:55.1799581Z [2022-04-23T02:27:07.931Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 989, in imwrite
2022-04-23T02:27:55.1799828Z [2022-04-23T02:27:07.931Z]     result = tif.write(data, shape=shape, dtype=dtype, **kwargs)
2022-04-23T02:27:55.1800086Z [2022-04-23T02:27:07.931Z] TypeError: write() got an unexpected keyword argument 'compress'
2022-04-23T02:27:55.1800217Z [2022-04-23T02:27:07.931Z] 
2022-04-23T02:27:55.1800431Z [2022-04-23T02:27:07.931Z] ======================================================================
2022-04-23T02:27:55.1800725Z [2022-04-23T02:27:07.931Z] ERROR: test_read_patches_cucim_2 (tests.test_lesion_froc.TestEvaluateTumorFROC)
2022-04-23T02:27:55.1800992Z [2022-04-23T02:27:07.931Z] ----------------------------------------------------------------------
2022-04-23T02:27:55.1801188Z [2022-04-23T02:27:07.931Z] Traceback (most recent call last):
2022-04-23T02:27:55.1801512Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 307, in setUp
2022-04-23T02:27:55.1801687Z [2022-04-23T02:27:07.931Z]     prepare_test_data()
2022-04-23T02:27:55.1802025Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 52, in prepare_test_data
2022-04-23T02:27:55.1802252Z [2022-04-23T02:27:07.931Z]     save_as_tif("temp_ground_truth_0", ground_truth)
2022-04-23T02:27:55.1802583Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 34, in save_as_tif
2022-04-23T02:27:55.1802829Z [2022-04-23T02:27:07.931Z]     imwrite(file_path, array, compress="jpeg", tile=(16, 16))
2022-04-23T02:27:55.1803177Z [2022-04-23T02:27:07.931Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 989, in imwrite
2022-04-23T02:27:55.1803423Z [2022-04-23T02:27:07.931Z]     result = tif.write(data, shape=shape, dtype=dtype, **kwargs)
2022-04-23T02:27:55.1803678Z [2022-04-23T02:27:07.931Z] TypeError: write() got an unexpected keyword argument 'compress'
2022-04-23T02:27:55.1803799Z [2022-04-23T02:27:07.931Z] 
2022-04-23T02:27:55.1804019Z [2022-04-23T02:27:07.931Z] ======================================================================
2022-04-23T02:27:55.1804310Z [2022-04-23T02:27:07.931Z] ERROR: test_read_patches_cucim_3 (tests.test_lesion_froc.TestEvaluateTumorFROC)
2022-04-23T02:27:55.1804572Z [2022-04-23T02:27:07.931Z] ----------------------------------------------------------------------
2022-04-23T02:27:55.1804766Z [2022-04-23T02:27:07.931Z] Traceback (most recent call last):
2022-04-23T02:27:55.1805130Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 307, in setUp
2022-04-23T02:27:55.1805309Z [2022-04-23T02:27:07.931Z]     prepare_test_data()
2022-04-23T02:27:55.1805650Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 52, in prepare_test_data
2022-04-23T02:27:55.1805876Z [2022-04-23T02:27:07.931Z]     save_as_tif("temp_ground_truth_0", ground_truth)
2022-04-23T02:27:55.1806204Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 34, in save_as_tif
2022-04-23T02:27:55.1806447Z [2022-04-23T02:27:07.931Z]     imwrite(file_path, array, compress="jpeg", tile=(16, 16))
2022-04-23T02:27:55.1806744Z [2022-04-23T02:27:07.931Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 989, in imwrite
2022-04-23T02:27:55.1806992Z [2022-04-23T02:27:07.931Z]     result = tif.write(data, shape=shape, dtype=dtype, **kwargs)
2022-04-23T02:27:55.1807250Z [2022-04-23T02:27:07.931Z] TypeError: write() got an unexpected keyword argument 'compress'
2022-04-23T02:27:55.1807372Z [2022-04-23T02:27:07.931Z] 
2022-04-23T02:27:55.1807592Z [2022-04-23T02:27:07.931Z] ======================================================================
2022-04-23T02:27:55.1807883Z [2022-04-23T02:27:07.931Z] ERROR: test_read_patches_cucim_4 (tests.test_lesion_froc.TestEvaluateTumorFROC)
2022-04-23T02:27:55.1808146Z [2022-04-23T02:27:07.931Z] ----------------------------------------------------------------------
2022-04-23T02:27:55.1808343Z [2022-04-23T02:27:07.931Z] Traceback (most recent call last):
2022-04-23T02:27:55.1808664Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 307, in setUp
2022-04-23T02:27:55.1808836Z [2022-04-23T02:27:07.931Z]     prepare_test_data()
2022-04-23T02:27:55.1809181Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 52, in prepare_test_data
2022-04-23T02:27:55.1809409Z [2022-04-23T02:27:07.931Z]     save_as_tif("temp_ground_truth_0", ground_truth)
2022-04-23T02:27:55.1809736Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 34, in save_as_tif
2022-04-23T02:27:55.1809980Z [2022-04-23T02:27:07.931Z]     imwrite(file_path, array, compress="jpeg", tile=(16, 16))
2022-04-23T02:27:55.1810276Z [2022-04-23T02:27:07.931Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 989, in imwrite
2022-04-23T02:27:55.1810522Z [2022-04-23T02:27:07.931Z]     result = tif.write(data, shape=shape, dtype=dtype, **kwargs)
2022-04-23T02:27:55.1810765Z [2022-04-23T02:27:07.931Z] TypeError: write() got an unexpected keyword argument 'compress'
2022-04-23T02:27:55.1810893Z [2022-04-23T02:27:07.931Z] 
2022-04-23T02:27:55.1811116Z [2022-04-23T02:27:07.931Z] ======================================================================
2022-04-23T02:27:55.1811455Z [2022-04-23T02:27:07.931Z] ERROR: test_read_patches_cucim_5 (tests.test_lesion_froc.TestEvaluateTumorFROC)
2022-04-23T02:27:55.1811717Z [2022-04-23T02:27:07.931Z] ----------------------------------------------------------------------
2022-04-23T02:27:55.1811913Z [2022-04-23T02:27:07.931Z] Traceback (most recent call last):
2022-04-23T02:27:55.1812238Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 307, in setUp
2022-04-23T02:27:55.1812415Z [2022-04-23T02:27:07.931Z]     prepare_test_data()
2022-04-23T02:27:55.1812754Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 52, in prepare_test_data
2022-04-23T02:27:55.1812981Z [2022-04-23T02:27:07.931Z]     save_as_tif("temp_ground_truth_0", ground_truth)
2022-04-23T02:27:55.1813354Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 34, in save_as_tif
2022-04-23T02:27:55.1813606Z [2022-04-23T02:27:07.931Z]     imwrite(file_path, array, compress="jpeg", tile=(16, 16))
2022-04-23T02:27:55.1813903Z [2022-04-23T02:27:07.931Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 989, in imwrite
2022-04-23T02:27:55.1814140Z [2022-04-23T02:27:07.931Z]     result = tif.write(data, shape=shape, dtype=dtype, **kwargs)
2022-04-23T02:27:55.1814390Z [2022-04-23T02:27:07.931Z] TypeError: write() got an unexpected keyword argument 'compress'
2022-04-23T02:27:55.1814520Z [2022-04-23T02:27:07.931Z] 
2022-04-23T02:27:55.1814743Z [2022-04-23T02:27:07.931Z] ======================================================================
2022-04-23T02:27:55.1815033Z [2022-04-23T02:27:07.931Z] ERROR: test_read_patches_cucim_6 (tests.test_lesion_froc.TestEvaluateTumorFROC)
2022-04-23T02:27:55.1815296Z [2022-04-23T02:27:07.931Z] ----------------------------------------------------------------------
2022-04-23T02:27:55.1815495Z [2022-04-23T02:27:07.931Z] Traceback (most recent call last):
2022-04-23T02:27:55.1815872Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 307, in setUp
2022-04-23T02:27:55.1816079Z [2022-04-23T02:27:07.931Z]     prepare_test_data()
2022-04-23T02:27:55.1816419Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 52, in prepare_test_data
2022-04-23T02:27:55.1816645Z [2022-04-23T02:27:07.931Z]     save_as_tif("temp_ground_truth_0", ground_truth)
2022-04-23T02:27:55.1816974Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 34, in save_as_tif
2022-04-23T02:27:55.1817217Z [2022-04-23T02:27:07.931Z]     imwrite(file_path, array, compress="jpeg", tile=(16, 16))
2022-04-23T02:27:55.1817504Z [2022-04-23T02:27:07.931Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 989, in imwrite
2022-04-23T02:27:55.1817753Z [2022-04-23T02:27:07.932Z]     result = tif.write(data, shape=shape, dtype=dtype, **kwargs)
2022-04-23T02:27:55.1818007Z [2022-04-23T02:27:07.932Z] TypeError: write() got an unexpected keyword argument 'compress'
2022-04-23T02:27:55.1818138Z [2022-04-23T02:27:07.932Z] 
2022-04-23T02:27:55.1818357Z [2022-04-23T02:27:07.932Z] ======================================================================
2022-04-23T02:27:55.1818645Z [2022-04-23T02:27:07.932Z] ERROR: test_read_patches_cucim_7 (tests.test_lesion_froc.TestEvaluateTumorFROC)
2022-04-23T02:27:55.1818906Z [2022-04-23T02:27:07.932Z] ----------------------------------------------------------------------
2022-04-23T02:27:55.1819101Z [2022-04-23T02:27:07.932Z] Traceback (most recent call last):
2022-04-23T02:27:55.1819422Z [2022-04-23T02:27:07.932Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 307, in setUp
2022-04-23T02:27:55.1819594Z [2022-04-23T02:27:07.932Z]     prepare_test_data()
2022-04-23T02:27:55.1819933Z [2022-04-23T02:27:07.932Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 52, in prepare_test_data
2022-04-23T02:27:55.1820220Z [2022-04-23T02:27:07.932Z]     save_as_tif("temp_ground_truth_0", ground_truth)
2022-04-23T02:27:55.1820548Z [2022-04-23T02:27:07.932Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 34, in save_as_tif
2022-04-23T02:27:55.1820787Z [2022-04-23T02:27:07.932Z]     imwrite(file_path, array, compress="jpeg", tile=(16, 16))
2022-04-23T02:27:55.1821081Z [2022-04-23T02:27:07.932Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 989, in imwrite
2022-04-23T02:27:55.1821326Z [2022-04-23T02:27:07.932Z]     result = tif.write(data, shape=shape, dtype=dtype, **kwargs)
2022-04-23T02:27:55.1821576Z [2022-04-23T02:27:07.932Z] TypeError: write() got an unexpected keyword argument 'compress'
2022-04-23T02:27:55.1821705Z [2022-04-23T02:27:07.932Z] 
2022-04-23T02:27:55.1821969Z [2022-04-23T02:27:07.932Z] ======================================================================
2022-04-23T02:27:55.1822263Z [2022-04-23T02:27:07.932Z] ERROR: test_read_patches_cucim_8 (tests.test_lesion_froc.TestEvaluateTumorFROC)
2022-04-23T02:27:55.1822526Z [2022-04-23T02:27:07.932Z] ----------------------------------------------------------------------
2022-04-23T02:27:55.1822721Z [2022-04-23T02:27:07.932Z] Traceback (most recent call last):
2022-04-23T02:27:55.1823042Z [2022-04-23T02:27:07.932Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 307, in setUp
2022-04-23T02:27:55.1823215Z [2022-04-23T02:27:07.932Z]     prepare_test_data()
2022-04-23T02:27:55.1823553Z [2022-04-23T02:27:07.932Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 52, in prepare_test_data
2022-04-23T02:27:55.1823778Z [2022-04-23T02:27:07.932Z]     save_as_tif("temp_ground_truth_0", ground_truth)
2022-04-23T02:27:55.1824101Z [2022-04-23T02:27:07.932Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 34, in save_as_tif
2022-04-23T02:27:55.1824345Z [2022-04-23T02:27:07.932Z]     imwrite(file_path, array, compress="jpeg", tile=(16, 16))
2022-04-23T02:27:55.1824640Z [2022-04-23T02:27:07.932Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 989, in imwrite
2022-04-23T02:27:55.1824884Z [2022-04-23T02:27:07.932Z]     result = tif.write(data, shape=shape, dtype=dtype, **kwargs)
2022-04-23T02:27:55.1825135Z [2022-04-23T02:27:07.932Z] TypeError: write() got an unexpected keyword argument 'compress'
2022-04-23T02:27:55.1825269Z [2022-04-23T02:27:07.932Z] 
2022-04-23T02:27:55.1825490Z [2022-04-23T02:27:07.932Z] ======================================================================
2022-04-23T02:27:55.1825779Z [2022-04-23T02:27:07.932Z] ERROR: test_read_patches_cucim_9 (tests.test_lesion_froc.TestEvaluateTumorFROC)
2022-04-23T02:27:55.1826048Z [2022-04-23T02:27:07.932Z] ----------------------------------------------------------------------
2022-04-23T02:27:55.1826246Z [2022-04-23T02:27:07.932Z] Traceback (most recent call last):
2022-04-23T02:27:55.1826568Z [2022-04-23T02:27:07.932Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 307, in setUp
2022-04-23T02:27:55.1826742Z [2022-04-23T02:27:07.932Z]     prepare_test_data()
2022-04-23T02:27:55.1827079Z [2022-04-23T02:27:07.932Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 52, in prepare_test_data
2022-04-23T02:27:55.1827296Z [2022-04-23T02:27:07.932Z]     save_as_tif("temp_ground_truth_0", ground_truth)
2022-04-23T02:27:55.1827623Z [2022-04-23T02:27:07.932Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 34, in save_as_tif
2022-04-23T02:27:55.1827866Z [2022-04-23T02:27:07.932Z]     imwrite(file_path, array, compress="jpeg", tile=(16, 16))
2022-04-23T02:27:55.1828164Z [2022-04-23T02:27:07.932Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 989, in imwrite
2022-04-23T02:27:55.1828457Z [2022-04-23T02:27:07.932Z]     result = tif.write(data, shape=shape, dtype=dtype, **kwargs)
2022-04-23T02:27:55.1828707Z [2022-04-23T02:27:07.932Z] TypeError: write() got an unexpected keyword argument 'compress'
2022-04-23T02:27:55.1828838Z [2022-04-23T02:27:07.932Z] 
2022-04-23T02:27:55.1829057Z [2022-04-23T02:27:07.932Z] ======================================================================
2022-04-23T02:27:55.1829315Z [2022-04-23T02:27:07.932Z] ERROR: test_read_malformats_0 (tests.test_wsireader.TestCuCIM)
2022-04-23T02:27:55.1829574Z [2022-04-23T02:27:07.932Z] ----------------------------------------------------------------------
2022-04-23T02:27:55.1829774Z [2022-04-23T02:27:07.932Z] Traceback (most recent call last):
2022-04-23T02:27:55.1830122Z [2022-04-23T02:27:07.932Z]   File "/opt/conda/lib/python3.8/site-packages/parameterized/parameterized.py", line 533, in standalone_func
2022-04-23T02:27:55.1830376Z [2022-04-23T02:27:07.932Z]     return func(*(a + p.args), **p.kwargs)
2022-04-23T02:27:55.1830720Z [2022-04-23T02:27:07.932Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_wsireader.py", line 194, in test_read_malformats
2022-04-23T02:27:55.1830908Z [2022-04-23T02:27:07.932Z]     file_path = save_gray_tiff(
2022-04-23T02:27:55.1831316Z [2022-04-23T02:27:07.932Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_wsireader.py", line 118, in save_gray_tiff
2022-04-23T02:27:55.1831588Z [2022-04-23T02:27:07.932Z]     imwrite(filename, img_gray, shape=img_gray.shape, photometric="rgb")
2022-04-23T02:27:55.1831891Z [2022-04-23T02:27:07.932Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 989, in imwrite
2022-04-23T02:27:55.1832140Z [2022-04-23T02:27:07.932Z]     result = tif.write(data, shape=shape, dtype=dtype, **kwargs)
2022-04-23T02:27:55.1832447Z [2022-04-23T02:27:07.932Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 2133, in write
2022-04-23T02:27:55.1832623Z [2022-04-23T02:27:07.932Z]     raise RuntimeError(
2022-04-23T02:27:55.1832997Z [2022-04-23T02:27:07.932Z] RuntimeError: None != 2 <StoredShape(frames=1, separate_samples=1, depth=1, length=16, width=16, contig_samples=1, extrasamples=0)>
2022-04-23T02:27:55.1833129Z [2022-04-23T02:27:07.932Z] 
2022-04-23T02:27:55.1833351Z [2022-04-23T02:27:07.932Z] ======================================================================
2022-04-23T02:27:55.1833614Z [2022-04-23T02:27:07.932Z] ERROR: test_read_malformats_0 (tests.test_wsireader.TestTiffFile)
2022-04-23T02:27:55.1833877Z [2022-04-23T02:27:07.932Z] ----------------------------------------------------------------------
2022-04-23T02:27:55.1834066Z [2022-04-23T02:27:07.932Z] Traceback (most recent call last):
2022-04-23T02:27:55.1834407Z [2022-04-23T02:27:07.932Z]   File "/opt/conda/lib/python3.8/site-packages/parameterized/parameterized.py", line 533, in standalone_func
2022-04-23T02:27:55.1834621Z [2022-04-23T02:27:07.932Z]     return func(*(a + p.args), **p.kwargs)
2022-04-23T02:27:55.1834975Z [2022-04-23T02:27:07.932Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_wsireader.py", line 194, in test_read_malformats
2022-04-23T02:27:55.1835165Z [2022-04-23T02:27:07.932Z]     file_path = save_gray_tiff(
2022-04-23T02:27:55.1835505Z [2022-04-23T02:27:07.932Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_wsireader.py", line 118, in save_gray_tiff
2022-04-23T02:27:55.1835777Z [2022-04-23T02:27:07.932Z]     imwrite(filename, img_gray, shape=img_gray.shape, photometric="rgb")
2022-04-23T02:27:55.1836074Z [2022-04-23T02:27:07.932Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 989, in imwrite
2022-04-23T02:27:55.1836320Z [2022-04-23T02:27:07.932Z]     result = tif.write(data, shape=shape, dtype=dtype, **kwargs)
2022-04-23T02:27:55.1836621Z [2022-04-23T02:27:07.932Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 2133, in write
2022-04-23T02:27:55.1836852Z [2022-04-23T02:27:07.932Z]     raise RuntimeError(
2022-04-23T02:27:55.1837223Z [2022-04-23T02:27:07.932Z] RuntimeError: None != 2 <StoredShape(frames=1, separate_samples=1, depth=1, length=16, width=16, contig_samples=1, extrasamples=0)>

@Nic-Ma
Copy link
Contributor

Nic-Ma commented Apr 23, 2022

BTW, I saw you added new test files as "XXX_new.py", would you rename the file name later to remove the "_new" postfix?

Thanks.

@Nic-Ma
Copy link
Contributor

Nic-Ma commented Apr 23, 2022

Hi @drbeh ,

Above CI tests error is because Tifffile package released a new version yesterday and has some breaking changes.
I slightly changed the CI test data and arg name, fixed it in PR: #4162.

Thanks.

@Nic-Ma
Copy link
Contributor

Nic-Ma commented Apr 23, 2022

/build

@bhashemian bhashemian merged commit 0cdec83 into Project-MONAI:dev Apr 23, 2022
wyli added a commit that referenced this pull request Apr 26, 2022
* 4095 Add bundle download (#4114)

* draft download

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

* update bundle download

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

* add url and load

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* rename args and remove a few places

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix flake8 issue

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

* enhance with reviews

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

* add instantiate for load

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

* fix black error

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

* add unittest

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

* add load to docs

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

* add skip

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

* add schemaerror

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

* fix partial places

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

* download zip bundle

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

* [DLMED] restore Exception for test

Signed-off-by: Nic Ma <nma@nvidia.com>

* update ts features

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

* add config_files test case

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

* enhance docstring example for args_file

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Nic Ma <nma@nvidia.com>

* Disable pylint error and fix CI tests of new tifffile (#4162)

* workaround

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] fix tifffile issue

Signed-off-by: Nic Ma <nma@nvidia.com>

* Fixed an error in DiNTS model implementation and enabled act and norm layer options (#4157)

* fixed a bug

Signed-off-by: dongy <dongy@nvidia.com>

* autofix

Signed-off-by: dongy <dongy@nvidia.com>

* update test case

Signed-off-by: dongy <dongy@nvidia.com>

Co-authored-by: dongy <dongy@nvidia.com>

* Split transform (#4153)

* Redesign whole slide image reading (#4107)

* Redesign BaseWSIReader,  WSIReader, CuCIMWSIReader

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Add unittests for WSIReader

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Add image mode for output validation

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update docs

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update references to new WSIReader

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Remove legacy WSIReader

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update unittests

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update docs

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* sort imports

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Clean up imports

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update docstrings

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update docs and docstrings

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix a typo

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Remove redundant checking

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update read and other methods

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update wsireader to support multi image and update docstrings

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Make workaround for CuImage objects

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Add unittests for multi image reading

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update a note about cucim

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update type hints and docstrings

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Implement Split transform

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Add unittests

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update formatting

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Implement SplitDict

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Add unittests for SplitDict

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Add docs

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Remove images from docs

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Address all comments

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Add example and size check

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update docs

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Revert references to new wsireader

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Add missing comma

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* fix bundle download test issue (#4169)

Signed-off-by: Yiheng Wang <vennw@nvidia.com>

* 4094 Enhance `ckpt_export` to save config files (#4159)

* [DLMED] enhance checkpoint export

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] update according to comments

Signed-off-by: Nic Ma <nma@nvidia.com>

* Move RGB/RGBA checks to base class (#4171)

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

Co-authored-by: Nic Ma <nma@nvidia.com>

* [CICD] To support temp dgx runner (#4175)

* Support new temp dgx runner

Signed-off-by: Peixin Li <pxli@nyu.edu>

* atol 1e-5

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

Co-authored-by: Wenqi Li <wenqil@nvidia.com>

* Test fix for AMP kwargs (#4178)

Signed-off-by: Eric Kerfoot <eric.kerfoot@kcl.ac.uk>

Co-authored-by: Yiheng Wang <68361391+yiheng-wang-nv@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Nic Ma <nma@nvidia.com>
Co-authored-by: dongyang0122 <don.yang.mech@gmail.com>
Co-authored-by: dongy <dongy@nvidia.com>
Co-authored-by: Behrooz <3968947+drbeh@users.noreply.github.com>
Co-authored-by: Peixin <pxli@nyu.edu>
Co-authored-by: Eric Kerfoot <17726042+ericspod@users.noreply.github.com>
Can-Zhao pushed a commit to Can-Zhao/MONAI that referenced this pull request May 10, 2022
* Redesign whole slide image reading (Project-MONAI#4107)

* Redesign BaseWSIReader,  WSIReader, CuCIMWSIReader

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Add unittests for WSIReader

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Add image mode for output validation

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update docs

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update references to new WSIReader

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Remove legacy WSIReader

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update unittests

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update docs

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* sort imports

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Clean up imports

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update docstrings

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update docs and docstrings

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix a typo

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Remove redundant checking

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update read and other methods

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update wsireader to support multi image and update docstrings

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Make workaround for CuImage objects

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Add unittests for multi image reading

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update a note about cucim

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update type hints and docstrings

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Implement Split transform

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Add unittests

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update formatting

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Implement SplitDict

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Add unittests for SplitDict

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Add docs

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Remove images from docs

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Address all comments

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Add example and size check

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Update docs

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Revert references to new wsireader

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>

* Add missing comma

Signed-off-by: Behrooz <3968947+drbeh@users.noreply.github.com>
@bhashemian bhashemian deleted the split-transform branch February 22, 2023 15:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Refactor Split transform

3 participants