Initial framework of an ethos-u runtime backend by robell · Pull Request #501 · pytorch/executorch

robell · 2023-09-27T16:19:10Z

Added shell of runtime Arm Backend for Ethos-U

./backends/arm/cmake/build.sh
will pull the appropriate compiler and build the ethos-u driver and delegate

This is yet to be linked into the executorch runtime

This also needs to be linked into a higher level app with the .pte in SRAM.

This is for early feedback - assuming some fairly heavy changes from here.

Signed-off-by: Rob Elliott <robert.elliott@arm.com>

* basic build system amendments * toolchain file for baremetal platforms * use of ethos-u-core-driver submodule * some scripts to pull a compiler and test-build the backend Signed-off-by: Rob Elliott <robert.elliott@arm.com>

…M, this will be reworked as wrapped ethos buffers in .pte become available Signed-off-by: Rob Elliott <robert.elliott@arm.com>

* Added cmmss submodule dependency * Added command_stream.cpp into the build * Added the target to the build script Signed-off-by: Rob Elliott <robert.elliott@arm.com>

netlify · 2023-09-27T16:19:17Z

✅ Deploy Preview for resplendent-gnome-14e531 ready!

Name	Link
🔨 Latest commit	`7007120`
🔍 Latest deploy log	https://app.netlify.com/sites/resplendent-gnome-14e531/deploys/651ad0bf30e7700008821238
😎 Deploy Preview	https://deploy-preview-501--resplendent-gnome-14e531.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

* vela binaries returned from preprocess * Included in PTE captured from arm_tosa_e2e * currently assumes vela is on path Signed-off-by: Rob Elliott <robert.elliott@arm.com>

Signed-off-by: Rob Elliott <robert.elliott@arm.com>

* added a scratch block in the vela_bin to preallocate it * added a vela_bin reading routine into ArmBackendEthosU * set pointers passed to vela based on vela_bin Signed-off-by: Rob Elliott <robert.elliott@arm.com>

Also helps with c++11 compliance if we have less c++20 code esp when we don't use it.

* fix to encode and decode of vela_bin_stream block sizes * hardcoded input/output population to check operation behaviour * use manual.h to init and register backend Signed-off-by: Rob Elliott <robert.elliott@arm.com>

digantdesai

Feel free to ignore comments for now ;)

digantdesai · 2023-10-02T04:47:48Z

CMakeLists.txt

 endif()

+# Build Arm Baremetal backend
+option(EXECUTORCH_BUILD_ARM_BAREMETAL


Nice. If we are planning to have multiple delegate flavors - i.e. runtime logic - under arm name then we can do something like,

EXECUTORCH_BUILD_ARM && EXECUTORCH_BUILD_ARM_<RUNTIME_FLAVOR>

Suggested change

option(EXECUTORCH_BUILD_ARM_BAREMETAL

option(EXECUTORCH_BUILD_ARM

digantdesai · 2023-10-02T04:51:56Z

kernels/portable/cpu/op_add.cpp

-    ET_SWITCH_REAL_TYPES_AND(Bool, b_type, ctx, "add", CTYPE_B, [&]() {
-      ET_SWITCH_REAL_TYPES_AND(Bool, common_type, ctx, "add", CTYPE_IN, [&]() {
-        ET_SWITCH_REAL_TYPES_AND(Bool, out_type, ctx, "add", CTYPE_OUT, [&]() {
+//  ET_SWITCH_REAL_TYPES_AND(Bool, a_type, ctx, "add", CTYPE_A, [&]() {


Another alternative is that we can switch the CPU-only example to softmax, while keeping the delegate example to add so we don't have to deal with this.

Because there is no good short-term solution for this besides we write a custom add op in ET which overwrites this Portable::Add

digantdesai · 2023-10-02T04:52:19Z

kernels/portable/cpu/vec_ops.h

 #include <cstring>
 #include <numeric>
 #include <type_traits>
+#include <cinttypes>


will be merged soon #558

digantdesai · 2023-10-02T04:54:40Z

kernels/portable/functions.yaml

-    - arg_meta: null
-      kernel_name: torch::executor::acosh_out
-
 - op: add.out


This is temp hack we can do better i.e. supply a new file with one op as oppose to delete rest of the ops in the default one - I was just lazy here :p

digantdesai · 2023-10-02T04:55:21Z

manual.h

+namespace torch {
+namespace executor {
+    void manual_override();
+    void digant_add_out(torch::executor::KernelRuntimeContext & context, EValue** stack);


this was also for testing the manual.h hack - we don't need this

digantdesai · 2023-10-02T04:56:20Z

runtime/core/function_ref.h

-
-  /**
-   * Case 2: A plain function pointer.
+   * Case 1: A plain function pointer.


Nicer fix is almost ready - #554, #555

digantdesai · 2023-10-02T04:57:54Z

runtime/platform/target/Posix.cpp

    if (!initialized) {                                             \
-      fprintf(                                                      \
-          ET_LOG_OUTPUT_FILE,                                       \
+      printf(                                                       \


So this is because stderr is patched to garbage value at link time when I link libexecutorch.a - easiest solution is to copy this into core_platforms "app" and overwrite this weak symbol.

Long term plan can be to have platform/target/ArmBaremetal.cpp PAL layer.

digantdesai · 2023-10-02T04:59:59Z

backends/arm/arm_backend.py

+
+        tosaname = "out.tosa"
+        flatbuffer = tosa_fb.serialize()
+        f = open(os.path.join(tmpdir,tosaname), "wb")


nit: use with for this resource

digantdesai · 2023-10-02T05:08:30Z

backends/arm/arm_backend.py

+        np_path = os.path.join(tmpdir,"output","out_sg0_vela.npz")
+        blocks = b''
+        with np.load(np_path, allow_pickle=False) as data:
+            # Emit the NPZ regions as:


OK for now but this should be refactored into its own function to emit_block

digantdesai · 2023-10-02T05:12:20Z

backends/arm/cmake/arm-none-eabi-gcc.cmake

+
+# Compilation warnings
+add_compile_options(
+#    -Wall


we (ET runtime) want to enable these eventually (or at least be cool when someone enables them) but right now we are not ready I guess

Signed-off-by: Rob Elliott <robert.elliott@arm.com>

* Removed dependencies on aything but driver * moved to minimal invocation pattern Signed-off-by: Rob Elliott <robert.elliott@arm.com>

* currently assumes function signature * read relevant argument data from vela_bin Signed-off-by: Rob Elliott <robert.elliott@arm.com>

robell · 2023-10-06T10:01:50Z

replaced by #595

robell added 5 commits September 27, 2023 08:34

Add ethos-u-core-driver submodule

7f44e16

Signed-off-by: Rob Elliott <robert.elliott@arm.com>

Added shell of runtime Arm Backend for Ethos-U

c6755bf

* basic build system amendments * toolchain file for baremetal platforms * use of ethos-u-core-driver submodule * some scripts to pull a compiler and test-build the backend Signed-off-by: Rob Elliott <robert.elliott@arm.com>

First-pass ethos-u backend with an assumed flat program format in SRA…

6421ead

…M, this will be reworked as wrapped ethos buffers in .pte become available Signed-off-by: Rob Elliott <robert.elliott@arm.com>

fixed builds of ethos-u-core-driver

1f62acc

* Added cmmss submodule dependency * Added command_stream.cpp into the build * Added the target to the build script Signed-off-by: Rob Elliott <robert.elliott@arm.com>

Merge branch 'pytorch:main' into main

ced2e06

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 27, 2023

mergennachin requested review from dbort, digantdesai, kimishpatel and mergennachin September 27, 2023 16:26

robell and others added 14 commits September 28, 2023 12:19

Emit Ethos-U55 chunked binaries from preprocess

83a5e32

* vela binaries returned from preprocess * Included in PTE captured from arm_tosa_e2e * currently assumes vela is on path Signed-off-by: Rob Elliott <robert.elliott@arm.com>

added executorch to the build targets

93cfdc2

Signed-off-by: Rob Elliott <robert.elliott@arm.com>

Extended the delegate to read 'vela_bin_stream's

a38f080

* added a scratch block in the vela_bin to preallocate it * added a vela_bin reading routine into ArmBackendEthosU * set pointers passed to vela based on vela_bin Signed-off-by: Rob Elliott <robert.elliott@arm.com>

Merge branch 'pytorch:main' into main

a538747

[ET][Portable] Add int types header

8de6e92

[Executorch] Simplify FunctionRef to make it more portable

644eafc

Also helps with c++11 compliance if we have less c++20 code esp when we don't use it.

[WIP] headrify pte

b307c31

[NOT FOR LAND] Hacks for ET_LOG

52dc73c

[NOT FOR LAND] Hack op_add to reduce size

b2a431f

[NOT FOR LAND] Hacks for add minimal example

77e8eb0

[NOT FOR LAND] HACK for manual kernel registration

9b244b3

[NOT FOR LAND] Allow enabling logging in Release mode

1cabc63

[NOT FOR LAND][arm] setup for core_platform

2f10ee3

Fixes to make simple_add run on hardware

298fb22

* fix to encode and decode of vela_bin_stream block sizes * hardcoded input/output population to check operation behaviour * use manual.h to init and register backend Signed-off-by: Rob Elliott <robert.elliott@arm.com>

digantdesai reviewed Oct 2, 2023

View reviewed changes

robell added 3 commits October 2, 2023 09:39

tidied binary reading and moved to ET_LOG

08d71d9

Signed-off-by: Rob Elliott <robert.elliott@arm.com>

Simplified EthosU invocation code

189b04c

* Removed dependencies on aything but driver * moved to minimal invocation pattern Signed-off-by: Rob Elliott <robert.elliott@arm.com>

Basic ethos output copy to EValue

7007120

* currently assumes function signature * read relevant argument data from vela_bin Signed-off-by: Rob Elliott <robert.elliott@arm.com>

robell mentioned this pull request Oct 3, 2023

Initial framework of an ethos-u runtime backend #2 #595

Closed

robell closed this Oct 6, 2023

robell added the partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm label Jul 4, 2024

Gasoonjia pushed a commit that referenced this pull request Jul 30, 2024

remove macos-12 test (#501)

3c66c45

	option(EXECUTORCH_BUILD_ARM_BAREMETAL
	option(EXECUTORCH_BUILD_ARM

Conversation

robell commented Sep 27, 2023

Uh oh!

netlify bot commented Sep 27, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for resplendent-gnome-14e531 ready!

Uh oh!

digantdesai left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

robell commented Oct 6, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

netlify bot commented Sep 27, 2023 •

edited

Loading