[Relay][ONNX]Import ONNX model to inference failed

[aitest11]

When I use ONNX model to inference , such as from_onnx.py(https://docs.tvm.ai/tutorials/frontend/from_onnx.html), I find these issue :

Traceback (most recent call last):
File "from_onnx.py", line 98, in
**tvm_output = intrp.evaluate(sym)(tvm.nd.array(x.astype(dtype)), params).asnumpy()
......
%0 = divide(%Input73, %Constant377) //
%1 = nn.conv2d(%0, %Constant321, kernel_size=[5, 5]) // an internal invariant was violdated while typechecking your program [15:47:10] /tvm/src/lang/data_layout.cc:273: Check failed: src_shape.size() == src_axis.size() (0 vs. 4) :
;
%2 = reshape(%Constant318, newshape=[8, 1, 1]) //
%3 = reshape(%2, newshape=[8]) //
%4 = expand_dims(%3, axis=1, num_newaxis=2) //
%5 = add(%1, %4) //
%6 = nn.relu(%5) //
%7 = nn.max_pool2d(%6, pool_size=[2, 2], strides=[2, 2]) // an internal invariant was violdated while typechecking your program [15:47:10] /tvm/src/relay/op/nn/pooling.cc:73: Check failed: data != nullptr:
;
%8 = nn.conv2d(%7, %Constant340, kernel_size=[5, 5]) //
%9 = reshape(%Constant346, newshape=[16, 1, 1]) //
%10 = reshape(%9, newshape=[16]) //
%11 = expand_dims(%10, axis=1, num_newaxis=2) //
%12 = add(%8, %11) //
%13 = nn.relu(%12) //
%14 = nn.max_pool2d(%13, pool_size=[3, 3], strides=[3, 3]) // an internal invariant was violdated while typechecking your program [15:47:10] /tvm/src/relay/op/nn/pooling.cc:73: Check failed: data != nullptr:
;
%15 = reshape(%14, newshape=[1, 256]) //
%16 = reshape(%Constant312, newshape=[256, 10]) //
%17 = transpose(%16, axes=[1, 0]) //
%18 = nn.dense(%15, %17, units=None) //
%19 = reshape(%Constant367, newshape=[10]) //
%20 = add(%18, %19) //
%20

I don,t know what errors happened when I use it.

[Wheest]

Have encountered a similar issue myself, with the error src_shape.size() == src_axis.size() (0 vs. 4) : .

Couldn't find a solution on the Discourse last time I checked.

The error can be reproduced (at least on my systems) using this gist Jupyter notebook, which creates a PyTorch model, exports to onnx, then tries to run load and run using Relay. Followed the same tutorial.

[jroesch]

Which versions are you using? I was able to download the exact code from the tutorial and run it on my Macbook with no error using the current HEAD of TVM and the most recent version of ONNX.

[jroesch]

There are two bugs causing the issue in your notebook. The first bug is that the ONNX importer annotates a scalar shape when the shape is unknown which gives you a type incorrect Relay program.

The second bug is that you were providing the shape information for the wrong input, the data input is 0 not 1.

import torch
import torch.nn as nn
import torchvision
import torchvision.transforms as transforms
from torch.autograd import Variable
import numpy as np
import torch.onnx

import tvm
import tvm.relay as relay
from matplotlib import pyplot as plt
from tvm.contrib import graph_runtime
import onnx

class ConvNet(nn.Module):
    def __init__(self):
        super(ConvNet, self).__init__()
        self.layer1 = nn.Sequential(
            nn.Conv2d(in_c, num_filters, kernel_size=kdim,
                      stride=stride, padding=padding, bias=False))
    def forward(self, x):
        out = self.layer1(x)
        return out

# set hyperparams
in_c = 512
num_filters = 512
in_h, in_w = 28, 28
kdim = 3
groups=1
stride=1
padding=0
in_shape = (1, in_c, in_h, in_w)

test_input = Variable(torch.randn(1, in_c, in_h, in_w))

# create model, and export to onnx
save_path = '/tmp/'
fname = 'mymodel'

save_name = save_path + fname + '.onnx'
pytorch_model = ConvNet()
torch.onnx.export(pytorch_model, test_input, save_name)


# ### Load Model into TVM

onnx_model = onnx.load(save_name)
# Check that the IR is well formed
onnx.checker.check_model(onnx_model)
# Print a human readable representation of the graph
print(onnx.helper.printable_graph(onnx_model.graph))

test_input = np.random.rand(1, in_c, in_h, in_w)
dtype = 'float32'

x = test_input
ctx = tvm.cpu()

# Compile with relay
# Your bug is right here.
input_name = '0'
shape_dict = {input_name: x.shape}
target = 'llvm'

import pdb; pdb.set_trace()
sym, params = relay.frontend.from_onnx(onnx_model, shape_dict)
print(sym)
with relay.build_config(opt_level=1):
    intrps = relay.build_module.create_executor('graph', sym,
                                                tvm.cpu(0), target)

# Execute in tvm
tvm_output = intrps.evaluate(sym)(test_input, **params).asnumpy()

I will open a fix for the bug on our side, you should be able to run by changing the number.

[aitest11]

@jroesch at first ,thank you
I modify tvm code as following:
else:
947 self._num_input += 1
948 # tshape = self._shape[i_name] if i_name in self._shape else ()
949 if i_name in self._shape:
950 tshape = self._shape[i_name]
951 else:
952 print ("Must provide an input shape for {0}.".format(i_name))
953 raise ValueError("Must provide an input shape for {0}.".format(i_name))
954 if isinstance(self._dtype, dict):
955 dtype = self._dtype[i_name] if i_name in self._dtype else d_type

and I also modiy the:
input_name = '0'

But I still failed ad fllowing:
Must provide an input shape for 1.
Traceback (most recent call last):
File "from_onnx_untrained.py", line 106, in
sym, params = relay.frontend.from_onnx(onnx_model, shape_dict, 'float32')
File "/tvm/python/tvm/relay/frontend/onnx.py", line 1142, in from_onnx
sym, params = g.from_onnx(graph, opset)
File "/tvm/python/tvm/relay/frontend/onnx.py", line 953, in from_onnx
raise ValueError("Must provide an input shape for {0}.".format(i_name))
ValueError: Must provide an input shape for 1.

[jroesch]

I was able to run the above code on my machine, by just making the appropriate change. I don't understand where the stack smashing error comes from I have never seen it before.

It looks like you didn't update the input to be 1.

[jroesch]

This should be fixed in master, if you are still having an issue follow up.