2975 torch fgbgtoindices

[wyli]

Signed-off-by: Wenqi Li wenqil@nvidia.com

Fixes #2975

Description

• unravel_index is still in numpy, otherwise it supports both backends added both backends

Status

Ready

Types of changes

• Non-breaking change (fix or new feature that would not break existing functionality).
• New tests added to cover the changes.
• Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
• Quick tests passed locally by running ./runtests.sh --quick --unittests.
• In-line docstrings updated.
• Documentation updated, tested make html command in the docs/ folder.

[Nic-Ma]

pending the PR, I will test with spleen fast training and get back ASAP.

Thanks.

[Nic-Ma]

Hi @rijobro ,

BTW, have you implemented this feature locally in your branch? If yes, maybe you can share some comments here.

Thanks.

[Nic-Ma]

Mark: training time with this PR around 60s, and without this PR 56-58s.

Thanks.

[wyli]

Mark: training time with this PR around 60s, and without this PR 56-58s.

Thanks.

I looked into this issue, the root cause is that
coordinates were (1) converted into list of numbers:

MONAI/monai/transforms/utils.py

Line 427 in 28fb7d6

corrected_centers: List[int] = [c.item() if isinstance(c, torch.Tensor) else c for c in centers] # type: ignore

and (2) the numbers are converted into tensors:

MONAI/monai/transforms/croppad/array.py

Line 405 in 28fb7d6

roi_center = torch.as_tensor(roi_center, dtype=torch.int16)

Previously fg_indices, bg_indices were numpy arrays, so there's no step (1) conversion.

So I think the issue is not really in this PR, it's from the generate_pos_neg_label_crop_centers and SpatialCrop of RandCropByPosNegLabeld.

[Nic-Ma]

Hi @wyli ,

OK, thanks for your investigation!
Let me try to fix the issue in a separate PR ASAP.

Thanks.

[wyli]

Hi @wyli ,

OK, thanks for your investigation! Let me try to fix the issue in a separate PR ASAP.

Thanks.

thanks! please help fix the issue and we can merge this PR after that

[Nic-Ma]

Sure, sounds good to me. Investigating.

Thanks.

[rijobro]

This looks great, thanks!

[Nic-Ma]

I think we already found the root cause of the performance issue, no need to put the fg / bg indices on GPU as we actually need to move it to CPU and index image.
So I will merge this PR first, then create another PR to optimize the logic.

Thanks.