Error in integration tests.test_integration_gpu_customization #6207
Description
Describe the bug
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 43.30it/s]
2023-03-21 03:24:26,375 - INFO - BundleGen from https://github.com/Project-MONAI/research-contributions/releases/download/algo_templates/d0fa876d.tar.gz
algo_templates.tar.gz: 64.0kB [00:01, 49.8kB/s]
2023-03-21 03:24:27,693 - INFO - Downloaded: /tmp/tmp5nrcdfvu/algo_templates.tar.gz
2023-03-21 03:24:27,693 - INFO - Expected md5 is None, skip md5 check for file /tmp/tmp5nrcdfvu/algo_templates.tar.gz.
2023-03-21 03:24:27,693 - INFO - Writing into directory: /tmp/tmphvljc4i6/workdir.
[info] gpu device 0 with minimum memory
[I 2023-03-21 03:24:27,874] A new study created in memory with name: no-name-3c8b3f2a-8a26-4ef9-9b1b-87d89840c905
patch_size [32, 32, 3]
patch_size_valid [32, 32, 3]
label_channels 1
monai.transforms.io.dictionary LoadImaged.__init__:image_only: Current default value of argument `image_only=False` has been deprecated since version 1.1. It will be changed to `image_only=True` in version 1.3.
Traceback (most recent call last):
File "/tmp/tmphvljc4i6/workdir/segresnet2d_0/scripts/dummy_runner.py", line 222, in <module>
fire.Fire(DummyRunnerSegResNet2D)
File "/opt/conda/lib/python3.10/site-packages/fire/core.py", line 141, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "/opt/conda/lib/python3.10/site-packages/fire/core.py", line 475, in _Fire
component, remaining_args = _CallAndUpdateTrace(
File "/opt/conda/lib/python3.10/site-packages/fire/core.py", line 691, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "/tmp/tmphvljc4i6/workdir/segresnet2d_0/scripts/dummy_runner.py", line 74, in __init__
image_shape = data_stat["stats_by_cases"][_k]["image_stats"]["shape"]
TypeError: 'NoneType' object is not subscriptable
[error] OOM
To Reproduce
/workspace/monai/MONAI# python -m tests.test_integration_gpu_customization
Expected behavior
The dummy runner can run for a few iterations.
Screenshots
The integration passes because it ignores error in the dummy runs.
Metadata
Metadata
Assignees
Labels
No labels