[7/10] Code generation for Pooling and Fully Connected via CMSIS-NN #9531

asparkhi · 2021-11-18T16:21:47Z

Support for Maxpool and AvgPool via CMSIS-NN.
Code generation for fully connected layer via CMSIS-NN

This PR depends on #9456.

Tracking issue #8646.

manupak

Similiar to other one. Seems like this contains contents from FC PR as well ?

src/relay/backend/contrib/cmsisnn/relay_to_tir.cc

manupak · 2021-12-02T15:57:22Z

src/relay/backend/contrib/cmsisnn/relay_to_tir.cc

+    tvm::Array<PrimExpr> context_buffer_args = {tir::StringImm(context_buffer_name),
+                                                ToArg(context_buffer_size)};
+
+    scalar_args = tvm::runtime::Concat(context_buffer_args, scalar_args);


I would suggest to keep the declaration/initiatlization vectors that get concat closer to the concat.

asparkhi · 2021-12-02T16:21:21Z

@manupa-arm I think we should merge this one as this contains both of those changes once I've finally pushed changes addressing your comments.

manupak · 2021-12-02T16:23:04Z

If we want to do that please close the other PR and adjust titles and descriptions of this PR to reflect its contents.

asparkhi · 2021-12-02T16:27:18Z

If we want to do that please close the other PR and adjust titles and descriptions of this PR to reflect its contents.

Done

Change-Id: I67f65e69dfa2c6d31c2ea928d92a827df80bc51a

Change-Id: Ibf22250d961a683208faee362d1960ea266347e8

Change-Id: I0675c3248515c9ddc6f25393a3756f907c98fd46

Change-Id: I2e9ca3edec85991be394bbdb2d1739c3afc62d5e

Change-Id: I38fbde65016713956371fa27dbd2a36f844f5799

manupak

Broadly looks good! There are few minor comments related to clarity which I dont want to block the PR for another CI cycle. Would you be able to address them in a follow up ?

manupak · 2021-12-06T17:10:06Z

src/relay/backend/contrib/cmsisnn/relay_to_tir.cc

                                        ToArg(clip_max)};

-    // cmsis_nn_dims *input_dims (NHWC)
+    // layout NHWC


Sorry Ashutosh, I dont still follow what this comment mean. How is this related to the following line ?

This was to help understand that the following shape is in NHWC layout which will be passed on to CMSIS-NN API eventually.

I think might worth saying that explicitly :)

manupak · 2021-12-06T17:10:40Z

src/relay/backend/contrib/cmsisnn/relay_to_tir.cc

-    Array<PrimExpr> input_dims = CMSISNNDimensions(input_shape);

-    // cmsis_nn_dims *filter_dims (OHWI for Conv2D and IHWO for depthwise)
+    // OHWI for Conv2D and IHWO for depthwise


Same here, where is the information OHWI is used ?

Same as above. Maybe I should explicitly say that this is for CMSIS-NN API?

I think the information is that the following line produces a tensor is that layout and subsequent access assume that ?

Yes, comment got displaced in the process of review and new edits 😆

manupak · 2021-12-06T17:12:41Z

src/relay/backend/contrib/cmsisnn/relay_to_tir.cc

@@ -194,7 +180,7 @@ class RelayToTIRVisitor : public MixedModeMutator {
    if (depth_multiplier != -1) {
      cmsisnn_api = "arm_depthwise_conv_wrapper_s8";
      Array<PrimExpr> depthwise_filter_shape{1, filter_shape[0], filter_shape[1], out_channels};


Is it possible to use something like
int kernel_pos_o = kernel_layout.find("O");
instead of the numbers ?

Yes, partly done above. Missed here.

manupak · 2021-12-06T17:15:36Z

src/relay/backend/contrib/cmsisnn/relay_to_tir.cc

+                                                ToArg(context_buffer_size)};
+
+    scalar_args = tvm::runtime::Concat(context_buffer_args, scalar_args);
+    scalar_args = tvm::runtime::Concat(scalar_args, cmsisnn_input_shape);


[Clarity] I think we can bring the declaration "cmsisnn_input_shape" closer to here as it is not used before this

We cannot. Its being used to derive another size in the middle. Also, I would prefer to keep concatenation of arguments at one place. Locating all arguments at same place has helped me with debugs.

I think Im missing something here :). Isn't it defined in line 291 and is there a use in between ?
Anyway, if it helps debugging, lets have it this way.

Sorry, I was looking at a different operator. For some of the others, context buffer creation requires this shape. Just to be consistent, I didn't want to change the code layout.

manupak · 2021-12-06T17:16:08Z

src/relay/backend/contrib/cmsisnn/relay_to_tir.cc

+
+    scalar_args = tvm::runtime::Concat(context_buffer_args, scalar_args);
+    scalar_args = tvm::runtime::Concat(scalar_args, cmsisnn_input_shape);
+    scalar_args = tvm::runtime::Concat(scalar_args, cmsisnn_filter_shape);


[Clarity] I think we can bring the declaration "cmsisnn_filter_shape" closer to here as it is not used before this

Same as above.

manupak · 2021-12-06T17:16:27Z

src/relay/backend/contrib/cmsisnn/relay_to_tir.cc

+    scalar_args = tvm::runtime::Concat(context_buffer_args, scalar_args);
+    scalar_args = tvm::runtime::Concat(scalar_args, cmsisnn_input_shape);
+    scalar_args = tvm::runtime::Concat(scalar_args, cmsisnn_filter_shape);
+    scalar_args = tvm::runtime::Concat(scalar_args, bias_shape);


[Clarity] I think we can bring the declaration "bias_shape" closer to here as it is not used before this

Same as above.

manupak · 2021-12-06T17:16:45Z

src/relay/backend/contrib/cmsisnn/relay_to_tir.cc

+    scalar_args = tvm::runtime::Concat(scalar_args, cmsisnn_input_shape);
+    scalar_args = tvm::runtime::Concat(scalar_args, cmsisnn_filter_shape);
+    scalar_args = tvm::runtime::Concat(scalar_args, bias_shape);
+    scalar_args = tvm::runtime::Concat(scalar_args, cmsisnn_output_shape);


[Clarity] I think we can bring the declaration "cmsisnn_output_shape" closer to here as it is not used before this

Same as above.

asparkhi · 2021-12-06T17:23:32Z

Thanks for the reviews.
Sure @manupa-arm, I can create a follow up for this. Please feel free to merge.

asparkhi · 2021-12-06T17:45:12Z

@manupa-arm Could you please put commit message as "Support for code generation of Maxpool, AvgPool and Fully Connected layers via CMSIS-NN"

manupak · 2021-12-06T17:46:40Z

Thanks @ashutosh-arm ! This is merged now.

…pache#9531) Support for code generation of Maxpool, AvgPool and Fully Connected layers via CMSIS-NN

asparkhi requested review from MarisaKirisame, ZihengJiang, anijain2305, areusch, comaniac, icemelon, jroesch, junrushao, jwfromm, leandron, manupak, mbaret, mbrookhart, merrymercy, slyubomirsky, tqchen, trevor-m, vinx13, wweic, yzhliu and zhiics as code owners November 18, 2021 16:21

asparkhi changed the title ~~Pooling~~ Code generation for Pooling via CMSIS-NN Nov 18, 2021

asparkhi changed the title ~~Code generation for Pooling via CMSIS-NN~~ [7/10] Code generation for Pooling via CMSIS-NN Nov 18, 2021

asparkhi mentioned this pull request Nov 18, 2021

[RFC][Tracking Issue] CMSIS-NN Integration #8646

Closed

asparkhi force-pushed the pooling branch from b06bb11 to 28bc3dd Compare December 1, 2021 09:49

manupak reviewed Dec 2, 2021

View reviewed changes

Code generation for fully connected layer via CMSIS-NN

093f3e1

Change-Id: I67f65e69dfa2c6d31c2ea928d92a827df80bc51a

Ashutosh Parkhi added 3 commits December 3, 2021 10:33

Code generation for Pooling layers via CMSIS-NN

2ea9511

Change-Id: Ibf22250d961a683208faee362d1960ea266347e8

Skip tests for i386 CI runs

c4d7c89

Change-Id: I0675c3248515c9ddc6f25393a3756f907c98fd46

Addressed review comments: removed disconcerting comments

a51669a

Change-Id: I2e9ca3edec85991be394bbdb2d1739c3afc62d5e

asparkhi force-pushed the pooling branch from 28bc3dd to a51669a Compare December 3, 2021 10:33

Dummy commit

a05b17d

Change-Id: I38fbde65016713956371fa27dbd2a36f844f5799

asparkhi changed the title ~~[7/10] Code generation for Pooling via CMSIS-NN~~ [7/10] Code generation for Pooling and Fully Connected via CMSIS-NN Dec 6, 2021

manupak approved these changes Dec 6, 2021

View reviewed changes

manupak merged commit 3f1c931 into apache:main Dec 6, 2021

asparkhi deleted the pooling branch December 7, 2021 10:05

asparkhi mentioned this pull request Dec 10, 2021

Follow up from CMSIS-NN Pooling failure #9708

Merged

ylc pushed a commit to ylc/tvm that referenced this pull request Jan 7, 2022

[7/10] Code generation for Pooling and Fully Connected via CMSIS-NN (a…

8bb072a

…pache#9531) Support for code generation of Maxpool, AvgPool and Fully Connected layers via CMSIS-NN

yangulei pushed a commit to yangulei/tvm that referenced this pull request Jan 11, 2022

[7/10] Code generation for Pooling and Fully Connected via CMSIS-NN (a…

8de4263

…pache#9531) Support for code generation of Maxpool, AvgPool and Fully Connected layers via CMSIS-NN

yangulei pushed a commit to yangulei/tvm that referenced this pull request Jan 12, 2022

[7/10] Code generation for Pooling and Fully Connected via CMSIS-NN (a…

ed48b7c

…pache#9531) Support for code generation of Maxpool, AvgPool and Fully Connected layers via CMSIS-NN

ylc pushed a commit to ylc/tvm that referenced this pull request Jan 13, 2022

[7/10] Code generation for Pooling and Fully Connected via CMSIS-NN (a…

574390d

…pache#9531) Support for code generation of Maxpool, AvgPool and Fully Connected layers via CMSIS-NN

qsqqsqqsq-intellif pushed a commit to qsqqsqqsq-intellif/tvm that referenced this pull request Apr 29, 2022

[7/10] Code generation for Pooling and Fully Connected via CMSIS-NN (a…

de361b8

…pache#9531) Support for code generation of Maxpool, AvgPool and Fully Connected layers via CMSIS-NN

driazati mentioned this pull request Jul 14, 2022

TVM v0.9.0.rc0 Release Candidate Notes #12102

Closed

[7/10] Code generation for Pooling and Fully Connected via CMSIS-NN #9531

[7/10] Code generation for Pooling and Fully Connected via CMSIS-NN #9531

Uh oh!

Conversation

asparkhi commented Nov 18, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

manupak left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

asparkhi commented Dec 2, 2021

Uh oh!

manupak commented Dec 2, 2021

Uh oh!

asparkhi commented Dec 2, 2021

Uh oh!

manupak left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

asparkhi commented Dec 6, 2021

Uh oh!

asparkhi commented Dec 6, 2021

Uh oh!

manupak commented Dec 6, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

asparkhi commented Nov 18, 2021 •

edited

Loading