Skip to content

[QNN][TFLite] Added support for fused-bias and quantized input in TRANSPOSE_CONV for TFLite. #6523

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
jainris wants to merge 3 commits into from
Closed

[QNN][TFLite] Added support for fused-bias and quantized input in TRANSPOSE_CONV for TFLite. #6523

jainris wants to merge 3 commits into from

Conversation

@jainris
Copy link
Contributor

@jainris jainris commented Sep 21, 2020

  • Added dilation_value attribute to dilate operator of Relay/TOPI.
    (Enables custom value for dilation, instead of always 0)
  • Added tests for dilation_value of dilate operator in Relay and TOPI.
  • Added support for quantized input in TRANSPOSE_CONV operator of TFLite.
  • Added tests for quantized input in TRANSPOSE_CONV operator of TFLite.

[QNN][TFLite] Added support for fused-bias and quantized input in TRA…
27b8012 
…NSPOSE_CONV for TFLite.

* Added dilation_value attribute to dilate operator of Relay/TOPI.
  (Enables custom value for dilation, instead of always 0)
* Added tests for dilation_value of dilate operator in Relay and TOPI.
* Added support for quantized input in TRANSPOSE_CONV operator of TFLite.
* Added tests for quantized input in TRANSPOSE_CONV operator of TFLite.
@jainris
Copy link
Contributor Author

jainris commented Sep 21, 2020

cc @anijain2305 @mbaret @u99127 @FrozenGene @tqchen

@anijain2305 anijain2305 self-assigned this Sep 21, 2020
mbaret
mbaret requested changes Sep 22, 2020
View reviewed changes
mbaret
mbaret requested changes Sep 23, 2020
View reviewed changes
Copy link
Contributor

@mbaret mbaret left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks almost there to me. Could we see if there's a hosted model somewhere with a transpose convolution in that we can test with? Also ping @giuseros as I know you're familiar with the maths behind this.

@mbaret
Copy link
Contributor

mbaret commented Sep 23, 2020

also ping @siju-samuel

@anijain2305
Copy link
Contributor

anijain2305 commented Sep 23, 2020 •
edited
Loading

Dilation part is good.

I am not sure about the conv2d transpose portion. My concern is that we now have to replicate the logic for different framework parsers. My suggestion would be to add qnn.conv2d_tranpose op and perform the "dilation + qnn.op.conv2d" lowering in QNN Legalize (example here - https://github.com/apache/incubator-tvm/blob/master/python/tvm/relay/qnn/op/legalizations.py#L266).

For now, we can make the transformation for all targets, not just specifically to ARM.

This will keep the option open to improve the schedule of conv2d_transpose as a whole if needed.

@ZihengJiang ZihengJiang added the status: need update need update based on feedbacks label Sep 23, 2020
@jainris
Copy link
Contributor Author

jainris commented Sep 24, 2020

Quantized Transpose Convolution code needs some changes, so bringing dilate operator changes independently in #6550.

@jainris jainris closed this Sep 24, 2020
giuseros pushed a commit to giuseros/incubator-tvm that referenced this pull request Nov 11, 2020
Add initial support for quantized transpose convolution in Relay
f5b23df 
This work is based on @jainris initial PR: apache#6523

I added a relay.qnn.conv2d_transpose node. The strategy I followed is to
convert to int16 and invoke nn.conv2d_transpose (which already exists in
relay). Main changes:

- The node declaration lives in relay/qnn/op/convolution_transpose.cc
- Cast int8->int16 and subsequent offset removal is in tvm/relay/qnn/op/legalizations.py.
- I added and tested the operator in the tflite front-end
- I added a unit-test in Relay for qnn.conv2d_transpose
giuseros pushed a commit to giuseros/incubator-tvm that referenced this pull request Nov 11, 2020
Add initial support for quantized transpose convolution in Relay
6da5514 
This work is based on @jainris initial PR: apache#6523

I added a relay.qnn.conv2d_transpose node. The strategy I followed is to
convert to int16 and invoke nn.conv2d_transpose (which already exists in
relay). Main changes:

- The node declaration lives in relay/qnn/op/convolution_transpose.cc
- Cast int8->int16 and subsequent offset removal is in tvm/relay/qnn/op/legalizations.py.
- I added and tested the operator in the tflite front-end
- I added a unit-test in Relay for qnn.conv2d_transpose

Co-authored-by: Rishabh Jain
giuseros pushed a commit to giuseros/incubator-tvm that referenced this pull request Nov 11, 2020
Add initial support for quantized transpose convolution in Relay
05c0d3d 
This work is based on @jainris initial PR: apache#6523

I added a relay.qnn.conv2d_transpose node. The strategy I followed is to
convert to int16 and invoke nn.conv2d_transpose (which already exists in
relay). Main changes:

- The node declaration lives in relay/qnn/op/convolution_transpose.cc
- Cast int8->int16 and subsequent offset removal is in tvm/relay/qnn/op/legalizations.py.
- I added and tested the operator in the tflite front-end
- I added a unit-test in Relay for qnn.conv2d_transpose

Co-authored-by: Rishabh Jain
giuseros pushed a commit to giuseros/incubator-tvm that referenced this pull request Nov 11, 2020
@jainris
Add initial support for quantized transpose convolution in Relay
75758f1 
This work is based on @jainris initial PR: apache#6523

I added a relay.qnn.conv2d_transpose node. The strategy I followed is to
convert to int16 and invoke nn.conv2d_transpose (which already exists in
relay). Main changes:

- The node declaration lives in relay/qnn/op/convolution_transpose.cc
- Cast int8->int16 and subsequent offset removal is in tvm/relay/qnn/op/legalizations.py.
- I added and tested the operator in the tflite front-end
- I added a unit-test in Relay for qnn.conv2d_transpose

Co-authored-by: Rishabh Jain <jainris@users.noreply.github.com>
@giuseros giuseros mentioned this pull request Nov 11, 2020
Merged
mbaret pushed a commit that referenced this pull request Nov 26, 2020
@jainris
Add initial support for quantized transpose convolution in Relay (#6899)
093629c 
* Add initial support for quantized transpose convolution in Relay

This work is based on @jainris initial PR: #6523

I added a relay.qnn.conv2d_transpose node. The strategy I followed is to
convert to int16 and invoke nn.conv2d_transpose (which already exists in
relay). Main changes:

- The node declaration lives in relay/qnn/op/convolution_transpose.cc
- Cast int8->int16 and subsequent offset removal is in tvm/relay/qnn/op/legalizations.py.
- I added and tested the operator in the tflite front-end
- I added a unit-test in Relay for qnn.conv2d_transpose

Co-authored-by: Rishabh Jain <jainris@users.noreply.github.com>

* Fix linting

* Addressing review comments

Co-authored-by: Rishabh Jain <jainris@users.noreply.github.com>
trevor-m pushed a commit to trevor-m/tvm that referenced this pull request Dec 2, 2020
@jainris
Add initial support for quantized transpose convolution in Relay (apa…
3f36198 
…che#6899)

* Add initial support for quantized transpose convolution in Relay

This work is based on @jainris initial PR: apache#6523

I added a relay.qnn.conv2d_transpose node. The strategy I followed is to
convert to int16 and invoke nn.conv2d_transpose (which already exists in
relay). Main changes:

- The node declaration lives in relay/qnn/op/convolution_transpose.cc
- Cast int8->int16 and subsequent offset removal is in tvm/relay/qnn/op/legalizations.py.
- I added and tested the operator in the tflite front-end
- I added a unit-test in Relay for qnn.conv2d_transpose

Co-authored-by: Rishabh Jain <jainris@users.noreply.github.com>

* Fix linting

* Addressing review comments

Co-authored-by: Rishabh Jain <jainris@users.noreply.github.com>
trevor-m pushed a commit to trevor-m/tvm that referenced this pull request Dec 4, 2020
@jainris
Add initial support for quantized transpose convolution in Relay (apa…
8dbc3cb 
…che#6899)

* Add initial support for quantized transpose convolution in Relay

This work is based on @jainris initial PR: apache#6523

I added a relay.qnn.conv2d_transpose node. The strategy I followed is to
convert to int16 and invoke nn.conv2d_transpose (which already exists in
relay). Main changes:

- The node declaration lives in relay/qnn/op/convolution_transpose.cc
- Cast int8->int16 and subsequent offset removal is in tvm/relay/qnn/op/legalizations.py.
- I added and tested the operator in the tflite front-end
- I added a unit-test in Relay for qnn.conv2d_transpose

Co-authored-by: Rishabh Jain <jainris@users.noreply.github.com>

* Fix linting

* Addressing review comments

Co-authored-by: Rishabh Jain <jainris@users.noreply.github.com>
trevor-m pushed a commit to neo-ai/tvm that referenced this pull request Dec 4, 2020
@jainris @trevor-m
Add initial support for quantized transpose convolution in Relay (apa…
79e6c8e 
…che#6899)

* Add initial support for quantized transpose convolution in Relay

This work is based on @jainris initial PR: apache#6523

I added a relay.qnn.conv2d_transpose node. The strategy I followed is to
convert to int16 and invoke nn.conv2d_transpose (which already exists in
relay). Main changes:

- The node declaration lives in relay/qnn/op/convolution_transpose.cc
- Cast int8->int16 and subsequent offset removal is in tvm/relay/qnn/op/legalizations.py.
- I added and tested the operator in the tflite front-end
- I added a unit-test in Relay for qnn.conv2d_transpose

Co-authored-by: Rishabh Jain <jainris@users.noreply.github.com>

* Fix linting

* Addressing review comments

Co-authored-by: Rishabh Jain <jainris@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mbaret mbaret mbaret requested changes

Assignees

@anijain2305 anijain2305

Labels

status: need update need update based on feedbacks

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@jainris @mbaret @anijain2305 @ZihengJiang