[Bug] CodeGenVM cannot handle this intrinsic now: Op(relax.nn.conv2d_transpose)

[sidsingla]

Hi,
I am getting the following error:

exec = tvm.compile(mod_from_torch, target="llvm")


*** tvm._ffi.base.TVMError: Traceback (most recent call last):
  7: _ZN3tvm7runtime13PackedFun
  6: tvm::runtime::TypedPackedFunc<tvm::IRModule (tvm::relax::ExecBuilder, tvm::IRModule)>::AssignTypedLambda<tvm::IRModule (*)(tvm::relax::ExecBuilder, tvm::IRModule)>(tvm::IRModule (*)(tvm::relax::ExecBuilder, tvm::IRModule), std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}::operator()(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*) const
  5: tvm::relax::relax_vm::VMCodeGen(tvm::relax::ExecBuilder, tvm::IRModule)
  4: tvm::relax::relax_vm::CodeGenVM::Run(tvm::relax::ExecBuilder, tvm::IRModule)
  3: tvm::relax::relax_vm::CodeGenVM::Codegen(tvm::relax::Function const&)
  2: tvm::relax::relax_vm::CodeGenVM::VisitExpr_(tvm::relax::SeqExprNode const*)
  1: tvm::relax::relax_vm::CodeGenVM::VisitExpr_(tvm::relax::CallNode const*)
  0: _ZN3tvm7runtime6deta
  File "/workspace/tvm/src/relax/backend/vm/codegen_vm.cc", line 156
TVMError: CodeGenVM cannot handle this intrinsic now:
Op(relax.nn.conv2d_transpose)

I am looking to convert a GAN model with Conv2d-transpose layers. This is after the TVM graph is generated from the Torch model directly.

Using v20.0 from the nightly build. Legalize() is used before the build
Thanks.

[kavin-sai-krishna]

Hi,
I've been debugging the issue and found that the legalization pass for conv2d_transpose fails with the following error:

"TOPI conv2d_transpose does not support input layout other than NCHW, and kernel layout other than IOHW, so cannot be legalized by TOPI"

This error comes from tvm/python/tvm/relax/transform/legalize_ops/nn.py. It seems the kernel layout provided by PyTorch don't match what TOPI expects.

To resolve this, I see two potential options:

1. Insert layout transformations in the frontend using R.op.nn.layout_transform to convert the kernel to IOHW before legalization.

2. Implement the InferLayout function for conv2d_transpose in tvm/src/relax/op/nn/convolution.cc, which is currently unimplemented.

@Hzfengsy @tlopex @mshr-h Would appreciate any suggestions or feedback on the best way to move forward.