[Bug][ONNX] CumSum crashed when call infer_value for axis

[jikechao]

For the simple ONNX model, It will crash when executed at this line, It seems that ONNXConverter can't parse the axis value.

model structure

Trackback

  File "/workplace/software/tvm/tvm_/python/tvm/driver/tvmc/frontends.py", line 169, in load
    return relay.frontend.from_onnx(model, shape=shape_dict, **kwargs)
  File "/workplace/software/tvm/tvm_/python/tvm/relay/frontend/onnx.py", line 7346, in from_onnx
    mod, params = g.from_onnx(graph, opset)
  File "/workplace/software/tvm/tvm_/python/tvm/relay/frontend/onnx.py", line 6963, in from_onnx
    self._construct_nodes(graph)
  File "/workplace/software/tvm/tvm_/python/tvm/relay/frontend/onnx.py", line 7078, in _construct_nodes
    op = self._convert_operator(op_name, inputs, attr, self.opset)
  File "/workplace/software/tvm/tvm_/python/tvm/relay/frontend/onnx.py", line 7204, in _convert_operator
    sym = convert_map[op_name](inputs, attrs, self._params)
  File "/workplace/software/tvm/tvm_/python/tvm/relay/frontend/onnx.py", line 2486, in _impl_v1
    dim = int(infer_value(dim, params).numpy())
  File "/workplace/software/tvm/tvm_/python/tvm/relay/frontend/common.py", line 547, in infer_value
    ), "All inputs to infer must be available in params."
AssertionError: All inputs to infer must be available in params.

Steps to reproduce

1. download the model cum_sum.onnx
2. compile the model: tvmc compile --target 'llvm' cum_sum.onnx

Triage

• frontend:onnx

cc @KJlaccHoeUM9l

@vvchernov @tqchen Could you take a look? is it an ONNXConverter bug?

[vvchernov]

Hello @jikechao! I see CI tests for ONNX front-end for cumsum, it looks like it should work, particularly it uses different axis. My opinion is that it is problem with params and I guess it is inside tvmc, but I'm not familiar with it

[jikechao]

@vvchernov Thanks for your comments!
It seems @leandron and @ekalda are very familiar with TVMC. Could you willing check this bug? Thanks a lot.

[ekalda]

Hi @jikechao, to me it looks like a problem with the model or ONNX frontend. It is failing an assert in

tvm/python/tvm/relay/frontend/common.py

Lines 545 to 547 in 608d357

	assert all(
	var.name_hint in params.keys() for var in analysis.free_vars(input_val)
	), "All inputs to infer must be available in params."

which checks that the unbound variable (in this case axis) has an associated value in params. So I assume it means that the value of axis has to be known at the compile time. Eyeballing the model in Netron it looks like there is indeed no numeric value associated with axis. If the expectation is that we can have an unknown axis during compile time, then that is a problem with ONNX frontend since params is extracted and axis value checked there :)

[vvchernov]

Thank you @ekalda for deeper analysis!

[vvchernov]

Hello @jikechao! In test it assumes that axis is constant. In your case it is defined at runtime, but this case is not implemented in ONNX front-end of TVM. You can extend it, but there is reasonable question: Do you know cases in real models with such behavior when axis is dynamic and defined at runtime inside model topology?

[jikechao]

Thank you @vvchernov @ekalda, The axis is a 1-dim variable, one of the inputs. the detail onnx model info is:

The model exported from the following code seems valid. Because this usage is learning from the onnx official documentation This should be a correct usage and common in the real model.

node = onnx.helper.make_node(
    "CumSum",
    inputs=["x", "axis"],
    outputs=["y"],
)