Initial framework of an ethos-u runtime backend #3 by robell · Pull Request #659 · pytorch/executorch

robell · 2023-10-06T11:59:28Z

Cleaned up ethos-u runtime flow, replaces #595

Added shell of runtime Arm Backend for Ethos-U

./backends/arm/cmake/build.sh will pull the appropriate compiler and build the ethos-u driver and delegate and produce the libraries needed to link an application in, which can run a delegated .pte with the revised vela_bin_stream form contained in these patches.

Currently supports .pte files stored in SRAM.

- Basic runtime targeting Corstone-300 with U55 - cross compile support with a cmake toolchain (Arm baremetal build) - support for a few models AoT -> TOSA -> Vela -> U55 hardware - dependencies on the ethos-u core driver and cmsis (submodules) Signed-off-by: Rob Elliott <robert.elliott@arm.com>

Signed-off-by: Rob Elliott <robert.elliott@arm.com>

* There is a toolchain/linking issue mixing hard and soft float ABI which is causing the test to fail, but the structure is there and the delegate is registered. Signed-off-by: Rob Elliott <robert.elliott@arm.com>

* Override the default CPU in cmake which was causing a mixture of FPU and ABI flags to be passed to different compilation stages. * Updated fallback logging implementation in delegate app to fix sporadic crash Signed-off-by: Rob Elliott <robert.elliott@arm.com>

* uses the simple_add model to run through the AoT flow and the various debug options for looking at export. * produces a .pte file for runtime delegation on the ArmBackend for Ethos-U55 platforms Signed-off-by: Rob Elliott <robert.elliott@arm.com>

Signed-off-by: Rob Elliott <robert.elliott@arm.com>

* export the list of offset from AoT floq * appropriately copy inputs and ouputs to/from SRAM Signed-off-by: Rob Elliott <robert.elliott@arm.com>

Signed-off-by: Rob Elliott <robert.elliott@arm.com>

* File moved to new path in examples * fix arg handling and md5sum for setup scripts Signed-off-by: Rob Elliott <robert.elliott@arm.com>

netlify · 2023-10-06T11:59:32Z

✅ Deploy Preview for resplendent-gnome-14e531 canceled.

Name	Link
🔨 Latest commit	`709a688`
🔍 Latest deploy log	https://app.netlify.com/sites/resplendent-gnome-14e531/deploys/652072de622f4c00082b4740

* Introduction of BI_INT to have a small set of pure int tests * added either vela or tosa output from compilation * Fixed tosa e2e tests to use tosa output form * unit tests currently use tosa due to missing vela dependency * vela e2e testing runs with default compile flags and emits to .pte Signed-off-by: Rob Elliott <robert.elliott@arm.com>

digantdesai

I know since you are waiting I am submitting these as I go. Just finished looked at the AoT python code. And next looking at the runtime bits.

CMakeLists.txt

backends/arm/CMakeLists.txt

backends/arm/arm_backend.py

backends/arm/cmake/arm-none-eabi-gcc.cmake

Signed-off-by: Rob Elliott <robert.elliott@arm.com>

digantdesai

round 2 for delegate runtime

backends/arm/runtime/ArmBackendEthosU.cpp

digantdesai

I think this is it. Not sure if I missed anything, I am not the biggest fan of this GH UI :\

backends/arm/test/test_models.py

backends/arm/test/test_tosa.py

examples/arm/ethos-u-setup/core_platform/patches/0007-Add-delegate-runner-test.patch

examples/arm/run.sh

Signed-off-by: Rob Elliott <robert.elliott@arm.com>

digantdesai

Looks good to me thanks @robell - this was a difficult one.
Created Issue #666 as a follow up.

Signed-off-by: Rob Elliott <robert.elliott@arm.com>

facebook-github-bot · 2023-10-06T21:21:53Z

@digantdesai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

digantdesai · 2023-10-07T01:58:27Z

backends/arm/runtime/ArmBackendEthosU.cpp

+
+#include <cstring>
+#include <memory>
+#include <vector>


Let's try to see if we can get away without this std::container. It will be better from size and portability point of view. Other than input/output we aren't using it anywhere else so should be easy to get rid of it.

facebook-github-bot · 2023-10-07T03:11:30Z

@digantdesai merged this pull request in 78197a3.

Update support tables and explanatory text.

robell added 18 commits October 6, 2023 07:46

Fixed error messages on runtime init

a7aa848

Signed-off-by: Rob Elliott <robert.elliott@arm.com>

lintrunner cleanup

f95fead

Signed-off-by: Rob Elliott <robert.elliott@arm.com>

Add ArmBackend to example scripts

524afc6

Signed-off-by: Rob Elliott <robert.elliott@arm.com>

Add delegate test and FVP output

2bfdb5f

* There is a toolchain/linking issue mixing hard and soft float ABI which is causing the test to fail, but the structure is there and the delegate is registered. Signed-off-by: Rob Elliott <robert.elliott@arm.com>

Fix delegate runner patch

8267612

Generate pte for delegate test on the fly

f40ab5f

Signed-off-by: Rob Elliott <robert.elliott@arm.com>

Added support for variable input output patterns

d10d620

Signed-off-by: Rob Elliott <robert.elliott@arm.com>

Handle multiple delegate inputs with SRAM offsets

b812898

* export the list of offset from AoT floq * appropriately copy inputs and ouputs to/from SRAM Signed-off-by: Rob Elliott <robert.elliott@arm.com>

Add TOSA ref model and Vela dependencies

44a46a1

Signed-off-by: Rob Elliott <robert.elliott@arm.com>

Cleanup from lintrunner and other bits of tidyup

cb16e68

Signed-off-by: Rob Elliott <robert.elliott@arm.com>

Removed ethos u driver build and cmsis dependency

8aa80d0

Signed-off-by: Rob Elliott <robert.elliott@arm.com>

renamed lib ethos_u to executorch_delegate_ethos_u

3a5fd4f

Signed-off-by: Rob Elliott <robert.elliott@arm.com>

lintfix

e676620

Signed-off-by: Rob Elliott <robert.elliott@arm.com>

tidied delegate_runner output

4b1125e

Signed-off-by: Rob Elliott <robert.elliott@arm.com>

Fixed some merge issues

7afc5e4

* File moved to new path in examples * fix arg handling and md5sum for setup scripts Signed-off-by: Rob Elliott <robert.elliott@arm.com>

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 6, 2023

robell mentioned this pull request Oct 6, 2023

Initial framework of an ethos-u runtime backend #2 #595

Closed

digantdesai reviewed Oct 6, 2023

View reviewed changes

robell commented Oct 6, 2023

View reviewed changes

backends/arm/cmake/arm-none-eabi-gcc.cmake Outdated Show resolved Hide resolved

review fixes

6b7a18a

Signed-off-by: Rob Elliott <robert.elliott@arm.com>

digantdesai reviewed Oct 6, 2023

View reviewed changes

robell added 2 commits October 6, 2023 18:52

review feedback/improvements

c453e43

Signed-off-by: Rob Elliott <robert.elliott@arm.com>

tidy up example scripts

1917b54

Signed-off-by: Rob Elliott <robert.elliott@arm.com>

digantdesai mentioned this pull request Oct 6, 2023

Unify executorch_runner app to handle all types of ptes #666

Closed

digantdesai approved these changes Oct 6, 2023

View reviewed changes

robell added 3 commits October 6, 2023 20:06

further review comments

20bf2eb

Signed-off-by: Rob Elliott <robert.elliott@arm.com>

revised path for ethosu_minimal

468d6fc

Signed-off-by: Rob Elliott <robert.elliott@arm.com>

lintfix

709a688

Signed-off-by: Rob Elliott <robert.elliott@arm.com>

Jerry-Ge mentioned this pull request Oct 6, 2023

[TOSA] Replace Linear lowering of using Matmul with FullyConnected #616

Closed

tatwaichong mentioned this pull request Oct 6, 2023

Add initial lowering of aten.convolution to tosa.conv2d support #615

Closed

digantdesai reviewed Oct 7, 2023

View reviewed changes

digantdesai mentioned this pull request Oct 7, 2023

Removed std::vector container from the Arm delegate runtime code #677

Closed

facebook-github-bot closed this in 78197a3 Oct 7, 2023

facebook-github-bot added the Merged label Oct 7, 2023

robell added the partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm label Jul 4, 2024

Gasoonjia pushed a commit that referenced this pull request Jul 30, 2024

Update quantization.md (#659)

e9def32

Update support tables and explanatory text.

Conversation

robell commented Oct 6, 2023

Uh oh!

netlify bot commented Oct 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for resplendent-gnome-14e531 canceled.

Uh oh!

digantdesai left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

digantdesai left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

digantdesai left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

digantdesai left a comment

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Oct 6, 2023

Uh oh!

digantdesai Oct 7, 2023

Choose a reason for hiding this comment

Uh oh!

digantdesai Oct 7, 2023

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Oct 7, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

netlify bot commented Oct 6, 2023 •

edited

Loading