[RFC] 3D model support

[masahi]

There have been some interests in running 3D networks with TVM. Relevant discussions include:
https://discuss.tvm.ai/t/conv3d-is-supported-by-tvm/4118
https://discuss.tvm.ai/t/is-conv3d-op-in-tvm-on-schedule/974

This is a call for help on bringing 3D support to TVM. This will enable new application domains such as medical and other volmetric data, video, mesh & point cloud etc.

Operators that need to be adapted for 3D include convolution, pad, pooling, upsampling. The only complexity is convolution, other operators can be trivially updated by adding "depth" dimension.

For convolution, we may adapt existing autotvm conv2d template to 3D, or wait for @merrymercy's autoscheduler. For cuda backend, we can start by enabling conv 3d in cuDNN.

Please comment below if you are interested in this topic.

[DaniDeniz]

I am interested in running 3D networks!
Thank you

[JuanIsernGhosn]

That's a great idea. It would be a great advance to include support for 3D models.

[JohnnyHopp]

Looking forward to seeing this new feature!!!

[optima2005]

@masahi, for other ops, pad can support any dimentions. I am looking at pooling, it seems there would be some works, current pooling implementation only supports 2D. I am thinking to create a unified version as we did for convolution.

[masahi]

@optima2005 yes, I prefer a unified implementation. But that can potentially affect existing users who are using 2D only ops. Most people dont care about 3D, so if we generalize some ops for 3D and in the process introduce perf regression or other bugs to exising 2D users, they will get upset.

I think one of the requiments for adding 3D support is to introduce absolutely ZERO impact to existing 2d users. In that sense, I think creating seperate 3D implementation, which will involve some code duplication, seems not ideal but a safer bet. Can you comment? @tqchen @yzhliu @soiferj @vinx13

[optima2005]

@masahi This should be the correct approach. How about we will try to implement new functions with the unified manner but only being used by 3D. And keep the existing 2D ones untouched. Then we could transform them to be unified easily when we could have enough experience and data in the future.

[masahi]

Ok, as long as your implementation is not too complicated we can go that way.

[tqchen]

@masahi shall we close this issue as we have initial set of ops and open more specific ones?

[masahi]

@optima2005 it would be interesting to add test cases on translating 3d network end to end. I think tf frontend is ready for this.