[Bug] [ONNX] QLinearMatMul works with binary output only

[alnah005]

I'm using from_onnx to convert my model
dense_h32_w32_c3_sNone_pNone_kNone.pt_quant.onnx.zip into tvm relay. It looks like b_scale in QLinearMatMul expects a scalar scale not a vector of scales.
I'm not sure if this is a problem with onnx creating a vector for the weights, or if QLinearMatMul should support scale vectors.

Info about the model:

1. Input: (batch, channel, height, width) -> (1, 3, 32, 32)
2. Global Average pool: (batch, channel, 1, 1) -> (1, 3, 1, 1)
3. Reshape: (batch, channel) -> (1, 3)
4. Dense Layer: (batch, channel, 20) -> (3, 20)

   Dense Layer is where the error occurs

You might notice when you debug b_scale's shape that it has shape=20.

Expected behavior

No errors converting my model dense_h32_w32_c3_sNone_pNone_kNone.pt_quant.onnx.zip into tvm relay using from_onnx.

Actual behavior

assert num_elem == 1, "Cannot squeeze tensor shape {} to scalar form.".format(x_shape)
E       AssertionError: Cannot squeeze tensor shape (20,) to scalar form.

[alnah005]

Code to reproduce:

import onnx
from tvm import relay
m = onnx.load("dense_h32_w32_c3_sNone_pNone_kNone.pt_quant.onnx")
relay.frontend.from_onnx(m)

Error:

AssertionError: Cannot squeeze tensor shape (20,) to scalar form.

[yuanfz98]

Hello,
@cconvey The current implementation of QLinearMatMul in TVM may not support scale vectors. The behaviour for now is parsing the input scale as scalar. See PR #8952.

[cconvey]

Thanks @alnah005 @FZYUAN-1 for creating this Issue and pinging me!

As far as I can tell, the input model is valid ONNX, and you're bumping into a limitation in the current version of the ONNX->TVM importer code.

I'm afraid I don't have the bandwidth to (personally) fix this a the moment, but if someone else wants to create a PR I'd be very happy to review it.

[yuanfz98]

Thanks @alnah005 @FZYUAN-1 for creating this Issue and pinging me!

As far as I can tell, the input model is valid ONNX, and you're bumping into a limitation in the current version of the ONNX->TVM importer code.

I'm afraid I don't have the bandwidth to (personally) fix this a the moment, but if someone else wants to create a PR I'd be very happy to review it.

Thanks for your reply!
I will fix this in next several days. @alnah005 @denise-k