DynUNet different shape in eval mode?

[rijobro]

Describe the bug

Line 126 of test_dynunet.py has:

with torch.no_grad():
    blah

but when changing to:

net.eval()
with torch.no_grad():
    blah

the unit test fails with the following error:

  File "test_dynunet.py", line 129, in test_shape
    self.assertEqual(len(results), len(expected_shape))
AssertionError: 1 != 3

Spotted in this PR, which calls eval and no_grad together: #1384.

To Reproduce

1. run python tests/test_dynunet.py -- all ok
2. insert net.eval() at line 126 of test_dynunet.py
3. run python tests/test_dynunet.py -- error

[rijobro]

Apologies, from the DynUnet documentation:

deep_supervision: whether to add deep supervision head before output. Defaults to True.
If added, in training mode, the network will output not only the last feature maps
(after being converted via output block), but also the previous feature maps that come
from the intermediate up sample layers.

so I guess the behaviour is to be expected.

[wyli]

maybe we should improve the usability, currently, it's indeed a bit confusing #1263