[FRONTEND][ONNX] 'Shape' operator not implemented

[rajh619]

I have an Alexnet-ONNX model converted from Tensorflow which contains 'Shape' operator.
Unfortunately while loading in nnvm , it fails with NotImplementedError: Operator Shape not implemented.
and i couldn't find the same in nnvm.symbol (nnvm tensor operator api).
Basically 'Shape' operator (as specified in onnx operator List ) is to infer the shape of input tensor as 1D int64 output tensor .
Is there any alternative way to output the shape in 1D int64 tensor in onnx frontend ?

[srkreddy1238]

I have an active PR in tensor flow for Shape.

You could take ref and implement for ONNX.

[rajh619]

could you please share the link of your PR .

[srkreddy1238]

#1232

[rajh619]

@srkreddy1238

I have converted only a fragment of alexnet tf model into onnx, containing the 'Shape' operator for easy debugging.
Below is my ONNX graph :

graph tf2onnx (
  %Placeholder:0[FLOAT, 1x227x227x3]
) initializers (
  %conv1/weights:0[FLOAT, 96x3x11x11]
  %conv1/biases:0[FLOAT, 96]
) {
  %Conv2D__2:0 = Transpose[perm = [0, 3, 1, 2]](%Placeholder:0)
  %Conv2D:0 = Conv[dilations = [1, 1], kernel_shape = [11, 11], strides = [4, 4]](%Conv2D__2:0, %conv1/weights:0)
  %Conv2D__3:0 = Transpose[perm = [0, 2, 3, 1]](%Conv2D:0)
  %BiasAdd:0 = Add(%Conv2D__3:0, %conv1/biases:0)
  %Shape:0 = Shape(%Conv2D__3:0)
  %Reshape__4:0 = Cast[to = 7](%Shape:0)
  %Reshape:0 = Reshape(%BiasAdd:0, %Reshape__4:0)
  %conv1_1:0 = Relu(%Reshape:0)
  return %conv1_1:0
}

def _shape():
    def _impl(inputs, attr, params):
        input_shapes = attr['_input_shapes'][inputs[0]]

        # Fix the -1 dimensions to 1
        input_shapes[0] = [1 if x == -1 else x for x in input_shapes[0]]
        params[attr['_node_name']] = tvm.nd.array(input_shapes[0])

        return _sym.Variable(name=attr['_node_name'],
                             shape=params[attr['_node_name']].shape)
    return _impl

In my graph Attributes for 'Shape' is empty{} . I see no attr['_input_shapes'] for op_type 'Shape'.
So i get following error :

, in _convert_operator
    sym = convert_map[op_name](inputs, attrs, self._params)
  File "C:\Users\HRG\AppData\Roaming\Python\Python36\site-packages\nnvm-0.8.0-py3.6.egg\nnvm\frontend\onnx.py", line 419, in _impl
    input_shapes = attr['_input_shapes'][inputs[0]]
KeyError: '_input_shapes'

So what does " %Shape:0 = Shape(%Conv2D__3:0)" mean ? how to feed Conv2D__3:0 as input for 'Shape'.

[srkreddy1238]

You can't copy the function as it is, it's only a reference.

Can you share the ONNX graph ?

[rajh619]

Hi please find the attached onnx file
alexnet_fragment.zip

[srkreddy1238]

@rajh619

I checked the graph and it's a bit different from tensorflow.
Tensorflow frontend has all the out_shapes for all nodes available hence the Shape operator is computed at frontend and made it as a param to graph.

We may need Shape as an operator from compiler / topi in this case.

@tqchen
Earlier assumption of output_shapes availability doesn't hold true for all frontends.
Do we implement new operator Shape here ?

[srkreddy1238]

@rajh619

I further checked and here is my observation.

Tensorflow uses Shape operator to derive shape as a Tensor.
This Tensor is passed to next operators like Reshape as a Tensor argument for shape.

TVM Reshape operator doesn't support Tensor argument for shape (it's a tuple instead).
This shape operator is specific to tensorflow and onnx (not sure if onnx inherited from tensorflow for compatibility).

At this point I advice to use tensorflow frontend directly which handle these issues with _output_shapes until we have "Shape" and extended "Reshape" operator support in TVM.

[rajh619]

@srkreddy1238 Thanks for the explanation .
I will try tensorflow frontend .But am mostly interested in ONNX front end ,as it is the open format for any deep learning model .Also i think it helps users to convert their DL models implemented in DL frameworks where there is no frontend support available in NNVM (like Caffe2/CNTK/Chainer models etc.) to ONNX using converters

So maybe i have to wait for the shape operator to be available in TVM to try out this ONNX model !

[srkreddy1238]

I will PR the shape and reshape_like soon. I could enhance the tensor flow too with these.

[zwhdang]

@srkreddy1238
Thanks for your explanation.
I just export the super_resolution.onnx by the latest version of pytorch(ebae3f) according to "http://pytorch.org/tutorials/advanced/super_resolution_with_caffe2.html". When I try the "from_onnx.py" in tutorials of nnvm, it fails with NotImplementedError: Operator Shape not implemented too. So the 'Shape' operator may by a common demand for several ONNX using frameworks.

[srkreddy1238]

@zwhdang
I further analyzed and here are some challenges around shape.
Implementing Shape is not a challenge but, the next operation after Shape is tricky.

For example Reshape receiving the "to shape" as an input Tensor cannot pass through compilation process in current framework.

I have seen Shape operator in Tensorflow and may be inherited into ONNX to keep the compatibility. But with TVM it's a challenge.

I was able to workaround in tensorflow frontend as I have the output_shapes and input_shapes in the graph definition.

[zwhdang]

@srkreddy1238
I just try the pretrained model of pytorch, such as AlexNet and ResNet, and the "Shape problem" appears multiply in tvm-onnx-frontend. So do you have any good suggestion for loading pytorch-onnx models? Thanks.

[tqchen]

Seems most of the shape requirement was to achieve reshape_like, which we cold have a specialization path

[srkreddy1238]

I am thinking on below two approaches.

• We could have a an extra function in nnvm build_module some where after infer_shape and around optimize to convert Shape followed by Resize to drop Shape and update attributes for Reshape.

Or

• A new pass backed by a new node API like
  FConversion: which indicates converting this node to param node.
  FInferAttributes: which is called on dependent nodes to alter the attributes and drop this input.
  ** This is just an initial though to standardize this requirement, I am yet to check the feasibility for this. request to comment on these options.