add warp layer

[kate-sann5100]

Is your feature request related to a problem? Please describe.
Current repository does not support warping base on ddf, which is indispensable for registration.

Describe the solution you'd like
Add a Warp layer.

Describe alternatives you've considered
N/A

Additional context
DeepReg implementation of Warp layer

[tvercaut]

Note that this is related to #789, #189 and #95

Pytorch's grid-sample function provides a GPU implementation of linear interpolation for image resampling.

There is already an initial implementation of higher order resampling here but it needs more testing and use case examples:
https://github.com/Project-MONAI/MONAI/tree/master/monai/csrc/resample

[kate-sann5100]

Note that this is related to #789, #189 and #95

Pytorch's grid-sample function provides a GPU implementation of linear interpolation for image resampling.

There is already an initial implementation of higher order resampling here but it needs more testing and use case examples:
https://github.com/Project-MONAI/MONAI/tree/master/monai/csrc/resample

Thank you for the useful information. I will try to figure out the best way to integrate both pytorch and MONAI implementations into the layer.

[kate-sann5100]

It would be super helpful if anyone could answer the following questions about the MONAI implementation of resampling:

• Is it true that in grid_pull(input, grid), grid[n, d, h, w] specifies the delta_z, delta_y, delta_x displacements from the sample location to the target location ?
• What is the expected range of grid? Is it defined in the pixel_space, like defined in create_grid?

Thanks in advance. @tvercaut

[tvercaut]

@brudfors is probably th ebest person to advice on this implementation as it stemmed from https://github.com/balbasty/nitorch

[brudfors]

Hi @kate-sann5100

I am happy to try to answer your questions.

The grid is defined in the target/output space and should specify the coordinates to sample in the input image. The grid coordinates should be defined in the voxel space of the input image. Here is some sample code that may, or may not, shed some more light on how its used:

https://github.com/balbasty/nitorch/blob/master/demo/demo_spatial.ipynb

[kate-sann5100]

@brudfors Thank you for the information, that completely answers my questions.

[tvercaut]

By the way, @brudfors is there a computational time comparison somewhere for pytorch grid_sample vs MONAI/nitorch grid_pull in say 3D linear interpolation mode?

[brudfors]

Hi @tvercaut,

The below Colab notebook shows the speed-up doing GPU resampling compared to CPU:

https://colab.research.google.com/drive/1qICvEDn-p8RnmaG-9v0sZu__6OQafM4A?usp=sharing

it is quite substantial.

[tvercaut]

Thanks. I was more interested in GPU grid_sample (vanilla pytorch) vs GPU grid_pull (nitorch based).

[brudfors]

Oh yes, sorry, I added that comparison to the notebook as well: vanilla pytorch and nitorch seem to perform similarly, but there is probably a more thorough validation that could be done.

[tvercaut]

I am reopening this issue following the comment above and the note from @wyli here:
#1463 (review)

I would suggest only using grid_pull for interpolation orders strictly higher than linear as these are not supported by the vanilla grid_sample. For linear and nearest, unless I missed it, there is no benefit in using grid_pull over grid_sample, especially in view of the requirement for compilation.

@kate-sann5100: Can you adapt you warp layer accordingly?