[microTVM][tutorial] AOT host-driven tutorial with TFLite model #12182

mehrdadh · 2022-07-25T20:41:05Z

gromero

@mehrdadh Did a first pass and have a few comments on it.

gromero · 2022-07-25T21:26:15Z

gallery/how_to/work_with_microtvm/micro_aot.py

+
+This tutorial is showcasing microTVM host-driven AoT compilation with
+a TFLite model. This tutorial can be executed on a X86 CPU using C runtime (CRT)
+or on Zephyr plarform on a microcontroller that supports Zephyr platform.


gromero · 2022-07-25T21:34:31Z

gallery/how_to/work_with_microtvm/micro_aot.py

+# Import a TFLite model
+# ---------------------
+#
+# To begin with, download and import a TFLite model from TinyMLPerf models.


It's not actually importing directly from the MLPerf Tiny bench, i.e. from:

https://github.com/mlcommons/tiny/tree/master/benchmark/training/keyword_spotting/trained_models

Should this benchmark be mentioned just as the original source of this model?

Also I think the current name for the bench is MLPerf Tiny, not TinyMLPerf, like in the text found in the READM.md here? https://github.com/mlcommons/tiny

I added comment about the origin of the model and fixed MLPerf Tiny name.

gallery/how_to/work_with_microtvm/micro_aot.py

gromero · 2022-07-25T22:09:29Z

gallery/how_to/work_with_microtvm/micro_aot.py

+`Mehrdad Hessar <https://github.com/mehrdadh>`_,
+`Alan MacDonald <https://github.com/alanmacd>`_
+
+This tutorial is showcasing microTVM host-driven AoT compilation with


I think it should be mentioned, at least briefly, the benefits of the AOT executor or the scenarios where it helps -- in contrast to the Graph executor.

added few comments, please take another look.

guberti

Lots of good suggestions from @gromero - I've only got a few nits.

guberti · 2022-07-26T11:36:33Z

gallery/how_to/work_with_microtvm/micro_aot.py

+`Alan MacDonald <https://github.com/alanmacd>`_
+
+This tutorial is showcasing microTVM host-driven AoT compilation with
+a TFLite model. This tutorial can be executed on a X86 CPU using C runtime (CRT)


nit here and in a few other places:

Suggested change

a TFLite model. This tutorial can be executed on a X86 CPU using C runtime (CRT)

a TFLite model. This tutorial can be executed on a x86 CPU using C runtime (CRT)

gallery/how_to/work_with_microtvm/micro_aot.py

guberti · 2022-07-26T11:38:26Z

gallery/how_to/work_with_microtvm/micro_aot.py

+# with the compiled model in microTVM. To do this, we use Project API. We have defined
+# CRT and Zephyr microTVM template projects which are used for X86 CPU and Zephyr platforms
+# respectively.
+#


Remove this and leave a blank line here

Suggested change

#

guberti · 2022-07-26T11:41:24Z

gallery/how_to/work_with_microtvm/micro_aot.py

+# **Note:** By default this tutorial runs on X86 CPU using CRT, if you would like to run on Zephyr platform
+# you need to export `TVM_MICRO_USE_HW` environment variable.
+#
+use_physical_hw = bool(os.getenv("TVM_MICRO_USE_HW"))


We should mention the KWS dataset we're using, and probably ought to credit Google as the author (see https://ai.googleblog.com/2017/08/launching-speech-commands-dataset.html).

added that.

areusch

great comments all around, just a few suggestions to clarify text.

areusch · 2022-07-26T14:46:21Z

gallery/how_to/work_with_microtvm/micro_aot.py

+# -------------------
+#
+# Now we need to define the target, runtime and executor. In this tutorial, we focused on
+# using AOT host driven executor. We use the host micro target which is for running a model


Suggest to make this super clear:

# Use the C runtime (crt) and enable static linking by setting system-lib to True RUNTIME = Runtime("crt", {"system-lib": True}) # Simulate a microcontroller on the host machine. Uses the main() from src/runtime/crt/host/main.cc. To use physical hardware, replace "host" with something matching your hardware. See abc location for instructions. TARGET = tvm.target.target.micro("host") # Use the AOT executor rather than graph or vm executors. Don't use unpacked API or C calling style EXECUTOR = Executor("aot")

gallery/how_to/work_with_microtvm/micro_aot.py

mehrdadh

@gromero @guberti @areusch thanks for the great feedbacks! PTAL

mehrdadh · 2022-07-26T17:30:41Z

gallery/how_to/work_with_microtvm/micro_aot.py

+`Alan MacDonald <https://github.com/alanmacd>`_
+
+This tutorial is showcasing microTVM host-driven AoT compilation with
+a TFLite model. This tutorial can be executed on a X86 CPU using C runtime (CRT)


mehrdadh · 2022-07-26T17:31:04Z

gallery/how_to/work_with_microtvm/micro_aot.py

+`Mehrdad Hessar <https://github.com/mehrdadh>`_,
+`Alan MacDonald <https://github.com/alanmacd>`_
+
+This tutorial is showcasing microTVM host-driven AoT compilation with


added few comments, please take another look.

mehrdadh · 2022-07-26T17:31:17Z

gallery/how_to/work_with_microtvm/micro_aot.py

+
+This tutorial is showcasing microTVM host-driven AoT compilation with
+a TFLite model. This tutorial can be executed on a X86 CPU using C runtime (CRT)
+or on Zephyr plarform on a microcontroller that supports Zephyr platform.


mehrdadh · 2022-07-26T17:35:37Z

gallery/how_to/work_with_microtvm/micro_aot.py

+# Import a TFLite model
+# ---------------------
+#
+# To begin with, download and import a TFLite model from TinyMLPerf models.


I added comment about the origin of the model and fixed MLPerf Tiny name.

mehrdadh · 2022-07-26T17:37:03Z

gallery/how_to/work_with_microtvm/micro_aot.py

+# **Note:** By default this tutorial runs on X86 CPU using CRT, if you would like to run on Zephyr platform
+# you need to export `TVM_MICRO_USE_HW` environment variable.
+#
+use_physical_hw = bool(os.getenv("TVM_MICRO_USE_HW"))


added that.

mehrdadh · 2022-07-26T17:41:37Z

gallery/how_to/work_with_microtvm/micro_aot.py

+# -------------------
+#
+# Now we need to define the target, runtime and executor. In this tutorial, we focused on
+# using AOT host driven executor. We use the host micro target which is for running a model


gallery/how_to/work_with_microtvm/micro_aot.py

mehrdadh · 2022-07-26T20:20:21Z

@gromero thanks for the second review. I addressed your comments

gromero

LGTM @mehrdadh Thanks for addressing the comments. I'll just give some time so @areusch and @guberti can have a chance to ack the changes too and then I'll merge it.

gromero · 2022-07-26T20:54:38Z

gallery/how_to/work_with_microtvm/micro_aot.py

 # to use the compiled model with microTVM. To do this, we use Project API. We have defined
-# CRT and Zephyr microTVM template projects which are used for x86 CPU and Zephyr platforms
+# CRT and Zephyr microTVM template projects which are used for x86 CPU and Zephyr boards
 # respectively.


I'm not a native English speaker, but I think that ideally there must be a comma before "respectively", so it's up to you to add it or not :) I won't block on this nit, so just saying in case you need to re-spin the PR after some other review comment and if you can confirm that is indeed correct ;)

there should be, for some reason my eyes where seeing it there but I didn't actually put it there lol

haha I missed it previously too! :)

gromero · 2022-07-27T18:33:12Z

@mehrdadh I'm not sure what failed here and how to reproduce it locally. Hardly it's related to your PR, but I wonder if there is any CI issue filed for that error. Any idea? If not, maybe just ret-rigger the CI for now.

guberti · 2022-07-27T19:45:09Z

@mehrdadh @gromero I've also seen this issue on my PRs, and I believe it has been resolved. git rebase -i main will fix it.

areusch

LGTM for me, @guberti can take a look and then we can merge

gromero · 2022-07-28T15:29:00Z

@mehrdadh Could you please rebase?

mehrdadh · 2022-07-28T16:06:41Z

@gromero done!

guberti

LGTM!

…he#12182) * Add aot tutorial

github-actions bot requested review from areusch and gromero July 25, 2022 20:41

gromero requested changes Jul 25, 2022

View reviewed changes

gromero reviewed Jul 25, 2022

View reviewed changes

guberti reviewed Jul 26, 2022

View reviewed changes

areusch reviewed Jul 26, 2022

View reviewed changes

mehrdadh commented Jul 26, 2022

View reviewed changes

gromero requested changes Jul 26, 2022

View reviewed changes

gromero approved these changes Jul 26, 2022

View reviewed changes

areusch approved these changes Jul 27, 2022

View reviewed changes

mehrdadh added 4 commits July 28, 2022 09:06

Add aot tutorial init

e88b33b

fix lint

12b8db7

address comments

a276ffc

address comments

83124d9

mehrdadh force-pushed the micro/add_aot_host_tutorial branch from 9f939b5 to 83124d9 Compare July 28, 2022 16:06

guberti approved these changes Jul 28, 2022

View reviewed changes

mehrdadh merged commit ebbce64 into apache:main Jul 28, 2022

mehrdadh deleted the micro/add_aot_host_tutorial branch July 28, 2022 18:56

AndrewZhaoLuo mentioned this pull request Oct 4, 2022

TVM v0.10.0.rc0 Release Candidate Notes #12979

Closed

xinetzone pushed a commit to daobook/tvm that referenced this pull request Nov 25, 2022

[microTVM][tutorial] AOT host-driven tutorial with TFLite model (apac…

512704a

…he#12182) * Add aot tutorial

	a TFLite model. This tutorial can be executed on a X86 CPU using C runtime (CRT)
	a TFLite model. This tutorial can be executed on a x86 CPU using C runtime (CRT)

[microTVM][tutorial] AOT host-driven tutorial with TFLite model #12182

[microTVM][tutorial] AOT host-driven tutorial with TFLite model #12182

Uh oh!

Conversation

mehrdadh commented Jul 25, 2022

Uh oh!

gromero left a comment

Choose a reason for hiding this comment

Uh oh!

gromero Jul 25, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

guberti left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

areusch left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

mehrdadh left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mehrdadh commented Jul 26, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gromero left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gromero Jul 25, 2022 •

edited

Loading

mehrdadh commented Jul 26, 2022 •

edited

Loading

gromero left a comment •

edited

Loading