-
Notifications
You must be signed in to change notification settings - Fork 3.8k
[Relay][transform][SimplifyExpr] simplify adjacent muls and adds with constants #13213
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment.
Generated by tvm-bot |
|
@vinx13 This PR could also solve the issue about |
|
tvm/tests/python/relay/test_pass_partition_graph.py Lines 1086 to 1089 in 5c9066d
@masahi The tests above failed because the pattern matcher always matches the last two adds in conv-add[bias]-add[bn]-add[sum], but the last add don't have any constant inputs thus cannot be simplified. Do you have any idea to fix this?
In this PR, I generalize the |
|
This sounds tricky, since I'd expect the pattern matcher to be operating in a "bottom up" manner. Is it possible to add "constant-ness" condition in the pattern? Otherwise I can only think of adding conv2d into the |
|
The pattern with Take Is there a way to evaluate the |
|
Since we run tvm/src/relay/backend/utils.cc Lines 264 to 266 in f7f2cda
|
|
tvm/src/relay/transforms/simplify_expr.cc Lines 786 to 790 in de7f762
The iterations happen inside SimplifyExpr, we need to find a way to evaluate the const expr manually.
Yes, but only for the const expr already exists before |
|
@masahi Problem solved after applying |
… constants (apache#13213) * simplify adjacent muls and adds with constants * apply FoldConstant inside SimplifyExpr
… constants (apache#13213) * simplify adjacent muls and adds with constants * apply FoldConstant inside SimplifyExpr
This PR enables simplification and folding of a sub graph containing adjacent
muls andadds with constant inputs.Motivation
Workloads like densenet-121 has several partitions with
conv-bn-mul-add-relupattern, for example:Current transforms on this pattern are:
conv-bn-mul-add-reluas the original pattern.conv-mul-add-mul-add-reluasbnis expended tomul-add.conv-add-mul-add-reluas the firstmulis folded intoconv.As all the
muls andadds have constant second inputs, they should be folded to a singlemul-addand a preferred transform sequence should be:conv-bn-mul-add-reluas the original pattern.conv-mul-add-mul-add-reluasbnis expended tomul-add.conv-mul-add-reluasmuls andadds are folded to one singlemul-add.conv-add-reluasmulis folded intoconv.Solution
Actually, any series contain
muls andadds with constant inputs could be folded to one particularmul-add. Three rewrite rules are added to make this happen:mul-mul->muladd-add->addadd-mul->mul-addAs
SimplifyExprapply simplifications iteratively until no changes to the graph, anymulandaddseries could be rewritten to one singlemul,addormul-addwith one of the binary inputs could evaluates to a constant in the followingFoldConstantpass.