[microTVM]Gemmini code generation using microTVM #13770

fzi-peccia · 2023-01-12T11:56:40Z

Added integration to generate C code able to execute neural networks on the Gemmini accelerator. Information about this can be found on this post

tvm-bot · 2023-01-12T11:56:43Z

Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment.

cc @alanmacd, @gromero, @leandron, @mehrdadh, @mkatanbaf _{See #10317 for details}

_{Generated by tvm-bot}

mehrdadh · 2023-01-17T21:40:56Z

@fzi-peccia fzi-peccia FYI, I have plan to review this PR this week.

mehrdadh

I did a first pass.

mehrdadh · 2023-01-17T22:54:22Z

apps/microtvm/gemmini/README.md

@@ -0,0 +1,3 @@
+This directory contains code to create code for the Gemmini accelerator using microTVM. These tests are then executed on the Spike RISC-V ISA simulator.
+
+In order to use this correctly, the Spike simulator has to be installed. This can be done by following the steps found on the Chipyard repository.


Link to instruction is missing

mehrdadh · 2023-01-17T23:02:58Z

apps/microtvm/gemmini/template_project/src/makefiles/conv2d/Makefile

@@ -0,0 +1,68 @@
+include $(abs_top_srcdir)/Makefrag


Could you consolidate all the Makefiles to a Makefile.template and modify it based on the project type in generate_project step?

mehrdadh · 2023-01-18T00:03:45Z

apps/microtvm/gemmini/template_project/microtvm_api_server.py

+        with open(source_dir / "model.h", "w") as f:
+            f.write(model_h_template.substitute(template_values))
+
+    # Arduino ONLY recognizes .ino, .ccp, .c, .h


mehrdadh · 2023-01-18T00:04:28Z

apps/microtvm/gemmini/template_project/microtvm_api_server.py

+        """Changes all #include statements in project_dir to be relevant to their
+        containing file's location.
+
+        Arduino only supports includes relative to a file's location, so this


fix the function description

mehrdadh · 2023-01-18T00:06:11Z

cmake/modules/contrib/Gemmini.cmake

@@ -0,0 +1,117 @@
+if(USE_MICRO)


I think this should be a separate flag which is disabled by default, maybe use USE_GEMMINI

mehrdadh · 2023-01-18T01:07:27Z

python/tvm/contrib/gemmini/tutorials/single_operators/add-tutorial.ipynb

@@ -0,0 +1,395 @@
+{


tutorial files should move to somewhere under gallery/how_to/. Also you need to change the format to .py file and write it in sphinx format. Now we support notebook generation and google colab, so you can even add cells to install all the dependencies and run it in google colab

Thanks for the feedback, I am on it

cmake/modules/contrib/Gemmini.cmake

mehrdadh

I did another pass. Thanks @fzi-peccia!

apps/microtvm/gemmini/template_project/crt_config/crt_config.h

apps/microtvm/gemmini/template_project/microtvm_api_server.py

mehrdadh · 2023-03-13T18:08:19Z

apps/microtvm/gemmini/template_project/src/Makefile

@@ -0,0 +1,74 @@
+# Licensed to the Apache Software Foundation (ASF) under one


It is highly recommended to use CMakeFile instead of Makefile to make it cross-platform compatible.

apps/microtvm/gemmini/template_project/src/add.c

python/tvm/contrib/gemmini/__init__.py

gallery/tutorial/micro_gemmini_conv2d.py

mehrdadh · 2023-03-13T18:30:58Z

python/tvm/contrib/gemmini/helpers.py

+ENV = Environment.instance()
+
+
+def create_header_file(


why not reuse the existing create_header_file function in TVM?

I changed to use the standard create_header_file from tvm.micro.testing.utils, but I changed a line in it to generate a define, instead of a const. I think this should be a define, but if that is not the case, I will need to continue using my own create_header_file

gallery/tutorial/micro_gemmini_add.py

… `requantize` (apache#13578) * wip * hack to convert size-1 scale and zp tensors to scalar * fix to binary op fast path * check output zp * add assert * add comment * lint * clean up beta handling * use regular binary op only for 32 bit add (bias addition) * do float(beta) when we know that beta is not None * restore original beta handling code to avoid mul by 1 * add comment on overflow

This fixes `ecr_pull` so that `docker-images.ini` can be updated with Docker images from a previous CI run for testing purposes Example run: https://ci.tlcpack.ai/blue/organizations/jenkins/tvm-cortexm/detail/PR-13590/4/pipeline/#step-80-log-9

…e_write (apache#13510) Add optional consumer blocks to cache_write.

Fix get tm allow_missing check pos

* add baddbmm conversion * fix * suppress lint

* [OpenCL][CI] Enable OpenCL cpp tests in CI * Add building gtest for OpenCL in GPU build * Fix CI build * Change OpenCL cpp tests build approach * Fix lint * Try to enable test in CI * Update version of gpu docker image * Change script mod

…che#12684) [Relay] Bug fix in relay.squeeze function. Also added functionality for parameter axis of type int

The current implementation of `CombineParallelDense` is hardcoded to slice along the last axis after the combined dense. I hit an error using this pass on the stable diffusion UNet, since it has a combined group where the dense is followed by `expand_dims` which changes the slicing axis (see https://github.com/masahi/torchscript-to-tvm/blob/master/stable-diffusion/compile.py for repro) ``` %76 = concatenate(%74) /* ty=Tensor[(20160, 1280), float32] */; %79 = concatenate(%77) /* ty=Tensor[(20160), float32] */; %78 = nn.dense(%75, %76, units=20160) /* ty=Tensor[(2, 20160), float32] */; %80 = nn.bias_add(%78, %79, axis=-1) /* ty=Tensor[(2, 20160), float32] */; %81 = expand_dims(%80, axis=2) /* ty=Tensor[(2, 20160, 1), float32] */; %82 = expand_dims(%81, axis=3) /* ty=Tensor[(2, 20160, 1, 1), float32] */; ``` The correct way to generate `strided_slice`: ``` %84 = strided_slice(%82, begin=[0, 0, 0, 0], end=[-1, 320, -1, -1], strides=[1, 1, 1, 1], slice_mode="size", axes=None) /* ty=Tensor[(2, 320, 1, 1), float32] */; ``` As I documented in the code, this fix is probably not 100% fail-proof. I think this is a difficult problem, since it requires tracking how the original output-channel axis of the combined dense moves across shape-changing operations like `reshape /transpose / split`. But this is at least "more correct" than the current implementation, so I'm submitting this fix as is for now. With this fix, `CombineParallelDense` works successfully on the stable diffusion UNet, and it reduces the number of `nn.dense` from 184 to 100.

… buffer (apache#13605) * Fix PlanAndUpdateBufferAllocationLocation not visiting constant buffer * add comment

…ache#13414) Enable depthwise conv2d NHWC with HWIO kernel layout. The default kernel layout is HWOI, matched to previous behavior.

…che#13602) * Add support for SequenceAt and SplitToSequence to onnx importer * Formatting * Change keepdims comparison * Only unify non-tuples in If

…#13606) * introduce LowerToPrimFunc to lower Relay func to TIR prim func * add doc * expose to python * adding test * another minor doc update * Verify that the input is a primitive function

…CopyConstants scheduler (apache#13588) In Ethos-U, CopyConstants scheduler currently copies weights for all operators. But in Vela, there are a number of scenarios where the weights are not buffered in SRAM, and FullyConnected case is one of them.

* fixed test * fix flag for arduino

Pass `std::nullopt` to initialization of `PassBuilder` for `PGOOptions`. LLVM is moving away from its own `Optional` type to `std::optional`.

…13616) default_rng was introduced in numpy 1.19, which is not present even in Ubuntu 20.04 (it comes with 1.17.4).

…abase (apache#13611) [Metaschedule] Align get_top_k logic in MemoryDatabase and JSONDatabase

…ase (apache#13618) * fixed tensor core batch_matmul legalize for transpose_b = False case * add test * clean up

…che#13615) In the Relay Matmul shape relation, we are a little over enthusiastic about unifying dynamic shapes. If one of the shapes is static, it does not need to be unified. This change only rewrites dynamic shapes to required static constraints. * Remove overwriting of matmul shapes when they are static * Simplify nesting * Add shape check to dense tests.

[Frontend] [ONNX] Support sequence_lens of GRU. Support convert sequence_lens input of GRU.

* [ETHOSN] Add support for experimental compiler option The support library currently supports enabling the experimental cascading compiler option via an environment variable `FORCE_EXPERIMENTAL_COMPILER`. This commit exposes the ability to enable this option through TVMC.

…#13622) * Fix print round-tripable multi thread env binding * add unittest

fzi-peccia · 2023-03-27T06:26:15Z

Thanks for the feedback @mehrdadh. I will work on this changes this week and let you know when everything is applied

…mini-microtvm

fzi-peccia · 2023-04-18T06:18:20Z

Hi @mehrdadh, all tests have passed except these two:

cortexm/pr-head: some problem with zephyr and mlfperf-tiny, this has nothing to do with my changes.
gpu/pr-head: some problem building the tutorials documentation, do you know a workaround to solve this?

mehrdadh changed the title ~~Gemmini code generation using microTVM~~ [microTVM]Gemmini code generation using microTVM Jan 12, 2023

mehrdadh requested changes Jan 18, 2023

View reviewed changes

kimjungwow reviewed Mar 7, 2023

View reviewed changes

cmake/modules/contrib/Gemmini.cmake Show resolved Hide resolved

mehrdadh requested changes Mar 13, 2023

View reviewed changes

Krzysztof Parzyszek and others added 24 commits March 27, 2023 08:22

[Hexagon] Skip test if "onnx" module not available (apache#13585)

8bcef00

[TIR][Schedule] Support for specific consumer block targeting in cach…

3008e78

…e_write (apache#13510) Add optional consumer blocks to cache_write.

[LLVM] Fix get tm allow_missing check pos (apache#13591)

f0f23d1

Fix get tm allow_missing check pos

[Torch] Stable diffusion support (apache#13594)

ed52610

* add baddbmm conversion * fix * suppress lint

[Relay] Bug fix in relay.squeeze function for issue apache#12400 (apa…

4a5032b

…che#12684) [Relay] Bug fix in relay.squeeze function. Also added functionality for parameter axis of type int

[Fix] Task scheduler error prompt upon build/run failure (apache#13601)

4aa8c55

[TIR] Fix PlanAndUpdateBufferAllocationLocation not visiting constant…

c8ffabc

… buffer (apache#13605) * Fix PlanAndUpdateBufferAllocationLocation not visiting constant buffer * add comment

[Hexagon] Enable depthwise conv2d NHWC with an HWIO kernel layout (ap…

0d4a2cd

…ache#13414) Enable depthwise conv2d NHWC with HWIO kernel layout. The default kernel layout is HWOI, matched to previous behavior.

[Relay][Frontend][Onnx] SequenceAt and SplitToSequence Operators (apa…

c48c063

…che#13602) * Add support for SequenceAt and SplitToSequence to onnx importer * Formatting * Change keepdims comparison * Only unify non-tuples in If

[Relay][TIR] Add utility to lower Relay func to TIR prim func (apache…

e181045

…#13606) * introduce LowerToPrimFunc to lower Relay func to TIR prim func * add doc * expose to python * adding test * another minor doc update * Verify that the input is a primitive function

[microTVM][Zephyr] Fix TVMC test on hardware (apache#13598)

2024e63

* fixed test * fix flag for arduino

[LLVM] Use std::nullopt instead of llvm::None (apache#13617)

f566e61

Pass `std::nullopt` to initialization of `PassBuilder` for `PGOOptions`. LLVM is moving away from its own `Optional` type to `std::optional`.

[Hexagon] Switch from default_rng to random in Hexagon tests (apache#…

69e3509

…13616) default_rng was introduced in numpy 1.19, which is not present even in Ubuntu 20.04 (it comes with 1.17.4).

[Metaschedule] Aligning get_top_k logic in MemoryDatabase and JSONDat…

f04ec55

…abase (apache#13611) [Metaschedule] Align get_top_k logic in MemoryDatabase and JSONDatabase

[TOPI] Fix batch_matmul tensorcore legalize for transpose_b = False c…

1938273

…ase (apache#13618) * fixed tensor core batch_matmul legalize for transpose_b = False case * add test * clean up

[Frontend] [ONNX] Support sequence_lens of GRU (apache#13587)

dfdaab7

[Frontend] [ONNX] Support sequence_lens of GRU. Support convert sequence_lens input of GRU.

[TVMScript] Fix print round-tripable multi thread env binding (apache…

af59d45

…#13622) * Fix print round-tripable multi thread env binding * add unittest

fzi-peccia added 5 commits March 27, 2023 08:22

Pending pylint fixes

e340fab

Pending pylint fixes

99756b2

Docs fix

d7e6a93

Added missing license text

1eaaee0

Small lint fixes

60cbfd1

fzi-peccia added 21 commits March 27, 2023 08:50

Merge remote-tracking branch 'upstream/main' into gemmini-microtvm

3075026

Recommended changes for merge

d563c00

Merge branch 'gemmini-microtvm' of github.com:fzi-peccia/tvm into gem…

a96bcaf

…mini-microtvm

Fixed merge issues

6316f83

Fix merge

0315a7c

Fixed lint problem

0e84cfe

Merge remote-tracking branch 'upstream/main' into gemmini-microtvm

8ce1222

.utils does not exist!

2f0308f

Added docstrings to all Python files

81c82df

Lint fixes

0cecb86

Merge remote-tracking branch 'upstream/main' into gemmini-microtvm

f0e5f5c

Fixed merge error

d53013a

Small lint fix

17369a4

Test fix

777816d

Lint fix

cb3fdaa

Changed URL in tutorial to the standard one used in other tutorials

0ecb826

Lint fix

d498f88

Merge fix

aff88a1

Lint fix

fb6330f

Merge remote-tracking branch 'upstream/main' into gemmini-microtvm

ad8b883

Fix test

7236a8b

Merge branch 'main' into gemmini-microtvm

3b07a14

tqchen closed this Feb 6, 2025

		@@ -0,0 +1,3 @@
		This directory contains code to create code for the Gemmini accelerator using microTVM. These tests are then executed on the Spike RISC-V ISA simulator.

		In order to use this correctly, the Spike simulator has to be installed. This can be done by following the steps found on the Chipyard repository.

		@@ -0,0 +1,74 @@
		# Licensed to the Apache Software Foundation (ASF) under one

		ENV = Environment.instance()


		def create_header_file(

[microTVM]Gemmini code generation using microTVM #13770

[microTVM]Gemmini code generation using microTVM #13770

Uh oh!

Conversation

fzi-peccia commented Jan 12, 2023

Uh oh!

tvm-bot commented Jan 12, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mehrdadh commented Jan 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mehrdadh left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mehrdadh left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

fzi-peccia commented Mar 27, 2023

Uh oh!

fzi-peccia commented Apr 18, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

tvm-bot commented Jan 12, 2023 •

edited

Loading

mehrdadh commented Jan 17, 2023 •

edited

Loading