From d4c665a642b93dbda5a08ef2f7adcbaf0d1f090d Mon Sep 17 00:00:00 2001 From: Guang Yang Date: Fri, 6 Oct 2023 13:23:06 -0700 Subject: [PATCH] Restructuring demos (#531) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Summary: Pull Request resolved: https://github.com/pytorch/executorch/pull/531 ----- ## Confusion arising from the current code structure: Reference to current code structure: https://github.com/pytorch/executorch/tree/main/examples The current code structure under `executorch/examples` mixes different levels of detail. For instance, the directories `backend/`, `quantization/`, and `export/` are intended for component-level demo code, such as showcasing quantization/export/delegation workflows or providing sample backend/quantizer/partitioner implementations. Assembling them to create a AOT path should not belong to any of it. However, the current structure combines recipes and individual components: - All contents of `backend/` are exclusively tailored for [xnnpack](https://github.com/pytorch/executorch/tree/main/examples/backend). - [export+delegate+composite](https://github.com/pytorch/executorch/blob/main/examples/export/export_and_delegate.py) and [bundled program example](https://github.com/pytorch/executorch/blob/main/examples/export/export_bundled_program.py) are placed in `export/` - Different runtimes are placed haphazardly., with items like [executor_runner](https://github.com/pytorch/executorch/tree/main/examples/executor_runner), [bundled_executor_runner](https://github.com/pytorch/executorch/tree/main/examples/bundled_executor_runner), and [xnn_executor_runner](https://github.com/pytorch/executorch/tree/main/examples/backend) - Backend-specific demos are scattered in the top-level directory, such as [arm's demo](https://github.com/pytorch/executorch/tree/main/examples/arm) and [partitioner/quantizer demo](https://github.com/pytorch/executorch/tree/main/examples/example_quantizer_and_delegate) Issues that are not directly tie to code restructuring but will need to be addressed as well. **Non-functional code/demo** (fix or hide): - [Unit tests](https://github.com/pytorch/executorch/tree/main/examples/export/test) under the `export/` directory are not runnable in the OSS environment. - The [example_quantizer_and_delegate](https://github.com/pytorch/executorch/tree/main/examples/example_quantizer_and_delegate) is not functioning and should be rectified or hidden. - The [arm's demo](https://github.com/pytorch/executorch/tree/main/examples/arm) is not runnable directly w/o additional guide. ----- ## Proposed code structure: organize demos in a way that is easier for users to understand how pieces are put together to create an e2e experience. We can go with this "vertical" structure if we could come up and agree on guidelines of whether a new demo should be considered as "adding a new demo" or "extending an existing demo". The purpose is to prevent randomly dumping demos in the top-level executorch/examples folder with boilerplate code. While this doesn't violate the rule of structuring demos vertically for user understanding, , but it literally has no structure at all, and it will quickly grow of out our maintenance. In my opinion, it would make sense to consider the following cases as **"adding a new demo"**: 1. Demo of a new 1st/3rd party backend, e.g. portable (default backend), xnnpack, arm, coreml, etc. 2. Demo of toolings, e.g. selective build, productivity sdk, etc. 3. Target specific apps, e.g. playground apps for Android/iOS/Embedded The rests will be considered as **"extending an existing demo"**, for example: - Demo of composibitlity, e.g. delegate and composite, whole/partial graph lowering, custom ops - Demo of specific components, e.g. quantization workflow, 2-stage export flow, example quantizer/partitioner, etc. After PTC we will spend more efforts on expanding and polishing the two major 1st-party demos (`examples/portable/` & `examples/xnnpack/`) we own in the long run. Maybe `examples/sdk` as well. So with the guidelines, the new top-level structure will looks like: ``` executorch/examples/ ├── README.md # top-level description for demos and folder structures ├── models/ ├── xnnpack/ # 1p e2e ├── portable/ # 1p e2e ├── arm/ # 3p backend e2e ├── qualcomm/ # 3p backend e2e ├── apple/ # 3p backend e2e ├── third-party/ # for the short-term, we can assume a centralized place for all third-party libs required by demos, e.g. sam, llama, etc. ├── selective_build/ ├── sdk/ ├── demo-apps/ └── utils/ # future ``` If we zoom in to `examples/portable/`, it will look like: ``` executorch/examples/portable/ ├── executor_runner/ ├── bundled_executor_runner/ ├── custom_ops/ ├── scripts/ │ ├── export_and_delegate.py │ ├── export_bundled_program.py │ └── export.py ├── test/ ├── __init__.py ├── utils.py └── README.md ``` If we zoom in to `examples/xnnpack/`, it will look like: ``` executorch/examples/xnnpack/ ├── quantization/ │ ├── example.py │ ├── TARGETS │ ├── test_quantize.sh │ └── utils.py ├── __init__.py ├── aot_compiler.py └── README.md ``` Reviewed By: mergennachin Differential Revision: D49714823 fbshipit-source-id: 72c28d9f176b3efae9f1e21bb2ff06f3dfff0fce --- .ci/scripts/gather_test_models.py | 5 +- .ci/scripts/test.sh | 24 ++-- .ci/scripts/utils.sh | 2 +- .github/workflows/pull.yml | 4 +- .github/workflows/trunk.yml | 4 +- CMakeLists.txt | 6 +- .../example}/README.md | 0 .../example}/TARGETS | 6 +- .../example}/diagrams/delegate.png | Bin .../example}/diagrams/memory_permute.png | Bin .../example}/diagrams/quantize_delegate.png | Bin .../example}/example_backend.py | 4 +- .../example_backend_delegate_passes/TARGETS | 2 +- .../merge_to_dim_pass.py | 0 .../permute_memory_formats_pass.py | 4 +- .../example}/example_operators/TARGETS | 0 .../example_operators/adaptive_avg_pool2d.py | 6 +- .../example}/example_operators/add.py | 6 +- .../example}/example_operators/conv2d.py | 6 +- .../example}/example_operators/conv_relu.py | 6 +- .../example}/example_operators/dropout.py | 6 +- .../example}/example_operators/flatten.py | 6 +- .../example}/example_operators/linear.py | 6 +- .../example}/example_operators/op_base.py | 0 .../example}/example_operators/ops.py | 26 ++--- .../example}/example_operators/utils.py | 0 .../example}/example_partitioner.py | 8 +- .../example}/example_quantizer.py | 4 +- .../example}/test_example_delegate.py | 8 +- build/cmake_deps.toml | 4 +- docs/source/getting-started-setup.md | 12 +- docs/source/runtime-overview.md | 2 +- .../tutorials/00_setting_up_executorch.md | 12 +- docs/website/docs/tutorials/profiling.md | 2 +- .../website/docs/tutorials/selective_build.md | 10 +- examples/README.md | 107 ++++-------------- .../ExecuTorchDemo.xcodeproj/project.pbxproj | 0 .../xcshareddata/xcschemes/App.xcscheme | 0 .../Resources/Examples/Arctic fox.jpg | Bin .../Resources/Examples/Samoyed.jpg | Bin .../Resources/Examples/hot pot.jpg | Bin .../ExecuTorchDemo/Sources/App/App.swift | 0 .../Sources/App/CameraController.swift | 0 .../Sources/App/CameraPreview.swift | 0 .../App/ClassificationController.swift | 0 .../Sources/App/ContentView.swift | 0 .../Sources/App/CustomViews.swift | 0 .../ImageClassification.swift | 0 .../MobileNet/MobileNet-Bridging-Header.h | 0 .../Sources/MobileNet/MobileNetClassifier.h | 0 .../Sources/MobileNet/MobileNetClassifier.mm | 0 .../MobileNet/MobileNetClassifier.swift | 0 .../Sources/MobileNet/Module.cpp | 0 .../ExecuTorchDemo/Sources/MobileNet/Module.h | 0 .../Test/MobileNetClassifierTest.swift | 0 .../ios => demo-apps/apple_ios}/README.md | 28 ++--- examples/portable/README.md | 53 +++++++++ examples/portable/TARGETS | 17 +++ examples/{export => portable}/__init__.py | 0 .../{ => portable}/custom_ops/CMakeLists.txt | 19 ++-- examples/{ => portable}/custom_ops/README.md | 2 +- .../{backend => portable/custom_ops}/TARGETS | 0 .../{ => portable}/custom_ops/custom_ops.yaml | 0 .../{ => portable}/custom_ops/custom_ops_1.py | 2 +- .../custom_ops/custom_ops_1_out.cpp | 0 .../custom_ops/custom_ops_2.cpp | 0 .../{ => portable}/custom_ops/custom_ops_2.py | 8 +- .../custom_ops/custom_ops_2_out.cpp | 0 .../{ => portable}/custom_ops/targets.bzl | 0 .../custom_ops/test_custom_ops.sh | 16 +-- .../executor_runner}/TARGETS | 0 .../executor_runner/executor_runner.cpp | 0 .../executor_runner/targets.bzl | 4 +- examples/portable/scripts/TARGETS | 13 +++ .../scripts/export.py} | 6 +- .../scripts}/export_and_delegate.py | 6 +- examples/{export => portable}/test/TARGETS | 2 +- .../{export => portable}/test/test_export.py | 4 +- examples/{export => portable}/utils.py | 0 examples/recipes/xnnpack_optimization/TARGETS | 12 -- .../recipes/xnnpack_optimization/__init__.py | 9 -- .../recipes/xnnpack_optimization/models.py | 31 ----- examples/sdk/scripts/TARGETS | 13 +++ .../scripts}/export_bundled_program.py | 7 +- examples/selective_build/CMakeLists.txt | 7 +- examples/selective_build/targets.bzl | 10 +- .../selective_build/test_selective_build.sh | 12 +- examples/{backend => xnnpack}/README.md | 67 ++++++++--- examples/{executor_runner => xnnpack}/TARGETS | 0 examples/xnnpack/__init__.py | 34 ++++++ .../aot_compiler.py} | 7 +- examples/xnnpack/quantization/TARGETS | 26 +++++ .../{ => xnnpack}/quantization/example.py | 8 +- .../quantization/test_quantize.sh | 8 +- examples/{ => xnnpack}/quantization/utils.py | 0 examples/{backend => xnnpack}/targets.bzl | 34 ++++-- 96 files changed, 412 insertions(+), 351 deletions(-) rename {examples/example_quantizer_and_delegate => backends/example}/README.md (100%) rename {examples/example_quantizer_and_delegate => backends/example}/TARGETS (81%) rename {examples/example_quantizer_and_delegate => backends/example}/diagrams/delegate.png (100%) rename {examples/example_quantizer_and_delegate => backends/example}/diagrams/memory_permute.png (100%) rename {examples/example_quantizer_and_delegate => backends/example}/diagrams/quantize_delegate.png (100%) rename {examples/example_quantizer_and_delegate => backends/example}/example_backend.py (81%) rename {examples/example_quantizer_and_delegate => backends/example}/example_backend_delegate_passes/TARGETS (78%) rename {examples/example_quantizer_and_delegate => backends/example}/example_backend_delegate_passes/merge_to_dim_pass.py (100%) rename {examples/example_quantizer_and_delegate => backends/example}/example_backend_delegate_passes/permute_memory_formats_pass.py (98%) rename {examples/example_quantizer_and_delegate => backends/example}/example_operators/TARGETS (100%) rename {examples/example_quantizer_and_delegate => backends/example}/example_operators/adaptive_avg_pool2d.py (88%) rename {examples/example_quantizer_and_delegate => backends/example}/example_operators/add.py (85%) rename {examples/example_quantizer_and_delegate => backends/example}/example_operators/conv2d.py (89%) rename {examples/example_quantizer_and_delegate => backends/example}/example_operators/conv_relu.py (89%) rename {examples/example_quantizer_and_delegate => backends/example}/example_operators/dropout.py (86%) rename {examples/example_quantizer_and_delegate => backends/example}/example_operators/flatten.py (84%) rename {examples/example_quantizer_and_delegate => backends/example}/example_operators/linear.py (90%) rename {examples/example_quantizer_and_delegate => backends/example}/example_operators/op_base.py (100%) rename {examples/example_quantizer_and_delegate => backends/example}/example_operators/ops.py (52%) rename {examples/example_quantizer_and_delegate => backends/example}/example_operators/utils.py (100%) rename {examples/example_quantizer_and_delegate => backends/example}/example_partitioner.py (94%) rename {examples/example_quantizer_and_delegate => backends/example}/example_quantizer.py (97%) rename {examples/example_quantizer_and_delegate => backends/example}/test_example_delegate.py (94%) rename examples/{apple/ios => demo-apps/apple_ios}/ExecuTorchDemo/ExecuTorchDemo.xcodeproj/project.pbxproj (100%) rename examples/{apple/ios => demo-apps/apple_ios}/ExecuTorchDemo/ExecuTorchDemo.xcodeproj/xcshareddata/xcschemes/App.xcscheme (100%) rename examples/{apple/ios => demo-apps/apple_ios}/ExecuTorchDemo/ExecuTorchDemo/Resources/Examples/Arctic fox.jpg (100%) rename examples/{apple/ios => demo-apps/apple_ios}/ExecuTorchDemo/ExecuTorchDemo/Resources/Examples/Samoyed.jpg (100%) rename examples/{apple/ios => demo-apps/apple_ios}/ExecuTorchDemo/ExecuTorchDemo/Resources/Examples/hot pot.jpg (100%) rename examples/{apple/ios => demo-apps/apple_ios}/ExecuTorchDemo/ExecuTorchDemo/Sources/App/App.swift (100%) rename examples/{apple/ios => demo-apps/apple_ios}/ExecuTorchDemo/ExecuTorchDemo/Sources/App/CameraController.swift (100%) rename examples/{apple/ios => demo-apps/apple_ios}/ExecuTorchDemo/ExecuTorchDemo/Sources/App/CameraPreview.swift (100%) rename examples/{apple/ios => demo-apps/apple_ios}/ExecuTorchDemo/ExecuTorchDemo/Sources/App/ClassificationController.swift (100%) rename examples/{apple/ios => demo-apps/apple_ios}/ExecuTorchDemo/ExecuTorchDemo/Sources/App/ContentView.swift (100%) rename examples/{apple/ios => demo-apps/apple_ios}/ExecuTorchDemo/ExecuTorchDemo/Sources/App/CustomViews.swift (100%) rename examples/{apple/ios => demo-apps/apple_ios}/ExecuTorchDemo/ExecuTorchDemo/Sources/ImageClassification/ImageClassification.swift (100%) rename examples/{apple/ios => demo-apps/apple_ios}/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/MobileNet-Bridging-Header.h (100%) rename examples/{apple/ios => demo-apps/apple_ios}/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/MobileNetClassifier.h (100%) rename examples/{apple/ios => demo-apps/apple_ios}/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/MobileNetClassifier.mm (100%) rename examples/{apple/ios => demo-apps/apple_ios}/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/MobileNetClassifier.swift (100%) rename examples/{apple/ios => demo-apps/apple_ios}/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/Module.cpp (100%) rename examples/{apple/ios => demo-apps/apple_ios}/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/Module.h (100%) rename examples/{apple/ios => demo-apps/apple_ios}/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/Test/MobileNetClassifierTest.swift (100%) rename examples/{apple/ios => demo-apps/apple_ios}/README.md (73%) create mode 100644 examples/portable/README.md create mode 100644 examples/portable/TARGETS rename examples/{export => portable}/__init__.py (100%) rename examples/{ => portable}/custom_ops/CMakeLists.txt (77%) rename examples/{ => portable}/custom_ops/README.md (95%) rename examples/{backend => portable/custom_ops}/TARGETS (100%) rename examples/{ => portable}/custom_ops/custom_ops.yaml (100%) rename examples/{ => portable}/custom_ops/custom_ops_1.py (94%) rename examples/{ => portable}/custom_ops/custom_ops_1_out.cpp (100%) rename examples/{ => portable}/custom_ops/custom_ops_2.cpp (100%) rename examples/{ => portable}/custom_ops/custom_ops_2.py (82%) rename examples/{ => portable}/custom_ops/custom_ops_2_out.cpp (100%) rename examples/{ => portable}/custom_ops/targets.bzl (100%) rename examples/{ => portable}/custom_ops/test_custom_ops.sh (80%) rename examples/{custom_ops => portable/executor_runner}/TARGETS (100%) rename examples/{ => portable}/executor_runner/executor_runner.cpp (100%) rename examples/{ => portable}/executor_runner/targets.bzl (92%) create mode 100644 examples/portable/scripts/TARGETS rename examples/{export/export_example.py => portable/scripts/export.py} (88%) rename examples/{export => portable/scripts}/export_and_delegate.py (98%) rename examples/{export => portable}/test/TARGETS (88%) rename examples/{export => portable}/test/test_export.py (98%) rename examples/{export => portable}/utils.py (100%) delete mode 100644 examples/recipes/xnnpack_optimization/TARGETS delete mode 100644 examples/recipes/xnnpack_optimization/__init__.py delete mode 100644 examples/recipes/xnnpack_optimization/models.py create mode 100644 examples/sdk/scripts/TARGETS rename examples/{export => sdk/scripts}/export_bundled_program.py (95%) rename examples/{backend => xnnpack}/README.md (58%) rename examples/{executor_runner => xnnpack}/TARGETS (100%) create mode 100644 examples/xnnpack/__init__.py rename examples/{backend/xnnpack_examples.py => xnnpack/aot_compiler.py} (94%) create mode 100644 examples/xnnpack/quantization/TARGETS rename examples/{ => xnnpack}/quantization/example.py (97%) rename examples/{ => xnnpack}/quantization/test_quantize.sh (85%) rename examples/{ => xnnpack}/quantization/utils.py (100%) rename examples/{backend => xnnpack}/targets.bzl (58%) diff --git a/.ci/scripts/gather_test_models.py b/.ci/scripts/gather_test_models.py index 3839f222ee7..c612af01358 100644 --- a/.ci/scripts/gather_test_models.py +++ b/.ci/scripts/gather_test_models.py @@ -10,7 +10,7 @@ from typing import Any from examples.models import MODEL_NAME_TO_MODEL -from examples.recipes.xnnpack_optimization import MODEL_NAME_TO_OPTIONS +from examples.xnnpack import MODEL_NAME_TO_OPTIONS # NB: Skip buck2 on MacOS to cut down the number of combinations we # need to run there as the number of MacOS runner is limited. Buck2 @@ -80,8 +80,7 @@ def export_models_for_ci() -> None: } delegation_configs = { False, - name in MODEL_NAME_TO_OPTIONS - and MODEL_NAME_TO_OPTIONS[name].xnnpack_delegation, + name in MODEL_NAME_TO_OPTIONS and MODEL_NAME_TO_OPTIONS[name].delegation, } for build_tool in BUILD_TOOLS.keys(): if target_os not in BUILD_TOOLS[build_tool]: diff --git a/.ci/scripts/test.sh b/.ci/scripts/test.sh index e8a39f8d282..b5a70361e0e 100755 --- a/.ci/scripts/test.sh +++ b/.ci/scripts/test.sh @@ -59,11 +59,11 @@ test_model() { cd ../../.. fi - "${PYTHON_EXECUTABLE}" -m examples.export.export_example --model_name="${MODEL_NAME}" + "${PYTHON_EXECUTABLE}" -m examples.portable.scripts.export --model_name="${MODEL_NAME}" # Run test model if [[ "${BUILD_TOOL}" == "buck2" ]]; then - buck2 run //examples/executor_runner:executor_runner -- --model_path "./${MODEL_NAME}.pte" + buck2 run //examples/portable/executor_runner:executor_runner -- --model_path "./${MODEL_NAME}.pte" elif [[ "${BUILD_TOOL}" == "cmake" ]]; then if [[ ! -f ${CMAKE_OUTPUT_DIR}/executor_runner ]]; then build_cmake_executor_runner @@ -98,24 +98,24 @@ test_model_with_xnnpack() { # Quantization-only if [[ ${WITH_QUANTIZATION} == true ]] && [[ ${WITH_DELEGATION} == false ]]; then - bash examples/quantization/test_quantize.sh "${BUILD_TOOL}" "${MODEL_NAME}" + bash examples/xnnpack/quantization/test_quantize.sh "${BUILD_TOOL}" "${MODEL_NAME}" exit 0 fi # Delegation if [[ ${WITH_QUANTIZATION} == true ]]; then SUFFIX="q8" - "${PYTHON_EXECUTABLE}" -m examples.backend.xnnpack_examples --model_name="${MODEL_NAME}" --delegate --quantize + "${PYTHON_EXECUTABLE}" -m examples.xnnpack.aot_compiler --model_name="${MODEL_NAME}" --delegate --quantize else SUFFIX="fp32" - "${PYTHON_EXECUTABLE}" -m examples.backend.xnnpack_examples --model_name="${MODEL_NAME}" --delegate + "${PYTHON_EXECUTABLE}" -m examples.xnnpack.aot_compiler --model_name="${MODEL_NAME}" --delegate fi OUTPUT_MODEL_PATH="${MODEL_NAME}_xnnpack_${SUFFIX}.pte" # Run test model if [[ "${BUILD_TOOL}" == "buck2" ]]; then - buck2 run //examples/backend:xnn_executor_runner -- --model_path "${OUTPUT_MODEL_PATH}" + buck2 run //examples/xnnpack:xnn_executor_runner -- --model_path "${OUTPUT_MODEL_PATH}" elif [[ "${BUILD_TOOL}" == "cmake" ]]; then if [[ ! -f ${CMAKE_OUTPUT_DIR}/backends/xnnpack/xnn_executor_runner ]]; then build_cmake_xnn_executor_runner @@ -129,15 +129,15 @@ test_model_with_xnnpack() { test_demo_backend_delegation() { echo "Testing demo backend delegation on AddMul" - "${PYTHON_EXECUTABLE}" -m examples.export.export_and_delegate --option "composite" - "${PYTHON_EXECUTABLE}" -m examples.export.export_and_delegate --option "partition" - "${PYTHON_EXECUTABLE}" -m examples.export.export_and_delegate --option "whole" + "${PYTHON_EXECUTABLE}" -m examples.portable.scripts.export_and_delegate --option "composite" + "${PYTHON_EXECUTABLE}" -m examples.portable.scripts.export_and_delegate --option "partition" + "${PYTHON_EXECUTABLE}" -m examples.portable.scripts.export_and_delegate --option "whole" # Run test model if [[ "${BUILD_TOOL}" == "buck2" ]]; then - buck2 run //examples/executor_runner:executor_runner -- --model_path "./composite_model.pte" - buck2 run //examples/executor_runner:executor_runner -- --model_path "./partition_lowered_model.pte" - buck2 run //examples/executor_runner:executor_runner -- --model_path "./whole.pte" + buck2 run //examples/portable/executor_runner:executor_runner -- --model_path "./composite_model.pte" + buck2 run //examples/portable/executor_runner:executor_runner -- --model_path "./partition_lowered_model.pte" + buck2 run //examples/portable/executor_runner:executor_runner -- --model_path "./whole.pte" elif [[ "${BUILD_TOOL}" == "cmake" ]]; then if [[ ! -f ${CMAKE_OUTPUT_DIR}/executor_runner ]]; then build_cmake_executor_runner diff --git a/.ci/scripts/utils.sh b/.ci/scripts/utils.sh index f0681d969a0..7a325ee351f 100644 --- a/.ci/scripts/utils.sh +++ b/.ci/scripts/utils.sh @@ -70,7 +70,7 @@ install_flatc_from_source() { build_executorch_runner_buck2() { # Build executorch runtime with retry as this step is flaky on macos CI - retry buck2 build //examples/executor_runner:executor_runner + retry buck2 build //examples/portable/executor_runner:executor_runner } build_executorch_runner_cmake() { diff --git a/.github/workflows/pull.yml b/.github/workflows/pull.yml index c8fb33ea617..3c73b09362c 100644 --- a/.github/workflows/pull.yml +++ b/.github/workflows/pull.yml @@ -58,7 +58,7 @@ jobs: MODEL_NAME=${{ matrix.model }} BUILD_TOOL=${{ matrix.build-tool }} XNNPACK_QUANTIZATION=${{ matrix.xnnpack_quantization }} - XNNPACK_DELEGATION=${{ matrix.xnnpack_delegation }} + XNNPACK_DELEGATION=${{ matrix.delegation }} DEMO_BACKEND_DELEGATION=${{ matrix.demo_backend_delegation }} PYTHON_EXECUTABLE=python bash .ci/scripts/setup-linux.sh "${BUILD_TOOL}" @@ -87,7 +87,7 @@ jobs: BUILD_TOOL=${{ matrix.build-tool }} PYTHON_EXECUTABLE=python bash .ci/scripts/setup-linux.sh "${BUILD_TOOL}" # Test custom ops - PYTHON_EXECUTABLE=python bash examples/custom_ops/test_custom_ops.sh "${BUILD_TOOL}" + PYTHON_EXECUTABLE=python bash examples/portable/custom_ops/test_custom_ops.sh "${BUILD_TOOL}" test-selective-build-linux: name: test-selective-build-linux diff --git a/.github/workflows/trunk.yml b/.github/workflows/trunk.yml index a38dbf9d764..f228b85babd 100644 --- a/.github/workflows/trunk.yml +++ b/.github/workflows/trunk.yml @@ -55,7 +55,7 @@ jobs: MODEL_NAME=${{ matrix.model }} BUILD_TOOL=${{ matrix.build-tool }} XNNPACK_QUANTIZATION=${{ matrix.xnnpack_quantization }} - XNNPACK_DELEGATION=${{ matrix.xnnpack_delegation }} + XNNPACK_DELEGATION=${{ matrix.delegation }} DEMO_BACKEND_DELEGATION=${{ matrix.demo_backend_delegation }} # Setup MacOS dependencies as there is no Docker support on MacOS atm @@ -85,7 +85,7 @@ jobs: PYTHON_EXECUTABLE=python bash .ci/scripts/setup-macos.sh "${BUILD_TOOL}" # Build and test custom ops - PYTHON_EXECUTABLE=python bash examples/custom_ops/test_custom_ops.sh "${BUILD_TOOL}" + PYTHON_EXECUTABLE=python bash examples/portable/custom_ops/test_custom_ops.sh "${BUILD_TOOL}" popd test-selective-build-macos: diff --git a/CMakeLists.txt b/CMakeLists.txt index 08cf06a6e5d..b5ae70dd4a4 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -93,8 +93,8 @@ set(CMAKE_CXX_FLAGS_DEBUG "-O0 -g") # Option to register custom operator `my_ops::mul3` or `my_ops::mul4` or no # custom ops at all. Custom ops are defined in -# `examples/custom_ops/custom_ops_1.py` and -# `examples/custom_ops/custom_ops_2.cpp`. +# `examples/portable/custom_ops/custom_ops_1.py` and +# `examples/portable/custom_ops/custom_ops_2.cpp`. option( REGISTER_EXAMPLE_CUSTOM_OP "Register whether custom op 1 (my_ops::mul3) or custom op 2 (my_ops::mul4) \ @@ -296,7 +296,7 @@ if(EXECUTORCH_BUILD_EXECUTOR_RUNNER) # Generate custom_ops_lib based on REGISTER_EXAMPLE_CUSTOM_OP if(REGISTER_EXAMPLE_CUSTOM_OP EQUAL 1 OR REGISTER_EXAMPLE_CUSTOM_OP EQUAL 2) - add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/examples/custom_ops) + add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/examples/portable/custom_ops) list(APPEND _executor_runner_libs custom_ops_lib) endif() diff --git a/examples/example_quantizer_and_delegate/README.md b/backends/example/README.md similarity index 100% rename from examples/example_quantizer_and_delegate/README.md rename to backends/example/README.md diff --git a/examples/example_quantizer_and_delegate/TARGETS b/backends/example/TARGETS similarity index 81% rename from examples/example_quantizer_and_delegate/TARGETS rename to backends/example/TARGETS index fa09a3053d5..1cbb37e1159 100644 --- a/examples/example_quantizer_and_delegate/TARGETS +++ b/backends/example/TARGETS @@ -8,7 +8,7 @@ python_library( ], deps = [ "//caffe2:torch", - "//executorch/examples/example_quantizer_and_delegate/example_operators:example_operators_lib", + "//executorch/backends/example/example_operators:example_operators_lib", ], ) @@ -18,7 +18,7 @@ python_library( "example_backend.py", ], deps = [ - "//executorch/examples/example_quantizer_and_delegate/example_backend_delegate_passes:lib", + "//executorch/backends/example/example_backend_delegate_passes:lib", "//executorch/exir/backend:backend_details", "//executorch/exir/backend:compile_spec_schema", ], @@ -32,7 +32,7 @@ python_library( deps = [ ":example_backend", "//caffe2:torch", - "//executorch/examples/example_quantizer_and_delegate/example_operators:example_operators_lib", + "//executorch/backends/example/example_operators:example_operators_lib", "//executorch/exir:graph_module", "//executorch/exir/backend:partitioner", "//executorch/exir/backend/canonical_partitioners:canonical_partitioner_lib", diff --git a/examples/example_quantizer_and_delegate/diagrams/delegate.png b/backends/example/diagrams/delegate.png similarity index 100% rename from examples/example_quantizer_and_delegate/diagrams/delegate.png rename to backends/example/diagrams/delegate.png diff --git a/examples/example_quantizer_and_delegate/diagrams/memory_permute.png b/backends/example/diagrams/memory_permute.png similarity index 100% rename from examples/example_quantizer_and_delegate/diagrams/memory_permute.png rename to backends/example/diagrams/memory_permute.png diff --git a/examples/example_quantizer_and_delegate/diagrams/quantize_delegate.png b/backends/example/diagrams/quantize_delegate.png similarity index 100% rename from examples/example_quantizer_and_delegate/diagrams/quantize_delegate.png rename to backends/example/diagrams/quantize_delegate.png diff --git a/examples/example_quantizer_and_delegate/example_backend.py b/backends/example/example_backend.py similarity index 81% rename from examples/example_quantizer_and_delegate/example_backend.py rename to backends/example/example_backend.py index 5b48f095d70..1652da5f26e 100644 --- a/examples/example_quantizer_and_delegate/example_backend.py +++ b/backends/example/example_backend.py @@ -7,10 +7,10 @@ import copy from typing import final, List -from executorch.examples.example_quantizer_and_delegate.example_backend_delegate_passes.merge_to_dim_pass import ( +from executorch.backends.example.example_backend_delegate_passes.merge_to_dim_pass import ( MergeToDimPass, ) -from executorch.examples.example_quantizer_and_delegate.example_backend_delegate_passes.permute_memory_formats_pass import ( +from executorch.backends.example.example_backend_delegate_passes.permute_memory_formats_pass import ( PermuteMemoryFormatsPass, ) diff --git a/examples/example_quantizer_and_delegate/example_backend_delegate_passes/TARGETS b/backends/example/example_backend_delegate_passes/TARGETS similarity index 78% rename from examples/example_quantizer_and_delegate/example_backend_delegate_passes/TARGETS rename to backends/example/example_backend_delegate_passes/TARGETS index 7e97876d57b..f4c8f4b85f2 100644 --- a/examples/example_quantizer_and_delegate/example_backend_delegate_passes/TARGETS +++ b/backends/example/example_backend_delegate_passes/TARGETS @@ -8,7 +8,7 @@ python_library( ], deps = [ "//caffe2:torch", - "//executorch/examples/example_quantizer_and_delegate/example_operators:example_operators_lib", + "//executorch/backends/example/example_operators:example_operators_lib", "//executorch/exir:dim_order_utils", "//executorch/exir:pass_base", "//executorch/exir/dialects:lib", diff --git a/examples/example_quantizer_and_delegate/example_backend_delegate_passes/merge_to_dim_pass.py b/backends/example/example_backend_delegate_passes/merge_to_dim_pass.py similarity index 100% rename from examples/example_quantizer_and_delegate/example_backend_delegate_passes/merge_to_dim_pass.py rename to backends/example/example_backend_delegate_passes/merge_to_dim_pass.py diff --git a/examples/example_quantizer_and_delegate/example_backend_delegate_passes/permute_memory_formats_pass.py b/backends/example/example_backend_delegate_passes/permute_memory_formats_pass.py similarity index 98% rename from examples/example_quantizer_and_delegate/example_backend_delegate_passes/permute_memory_formats_pass.py rename to backends/example/example_backend_delegate_passes/permute_memory_formats_pass.py index 2b1602f6aba..82cdb7a0506 100644 --- a/examples/example_quantizer_and_delegate/example_backend_delegate_passes/permute_memory_formats_pass.py +++ b/backends/example/example_backend_delegate_passes/permute_memory_formats_pass.py @@ -7,9 +7,7 @@ from itertools import chain import torch -from executorch.examples.example_quantizer_and_delegate.example_operators.ops import ( - module_to_annotator, -) +from executorch.backends.example.example_operators.ops import module_to_annotator from executorch.exir.dialects._ops import ops as exir_ops from executorch.exir.dim_order_utils import get_dim_order from executorch.exir.pass_base import ExportPass, PassResult diff --git a/examples/example_quantizer_and_delegate/example_operators/TARGETS b/backends/example/example_operators/TARGETS similarity index 100% rename from examples/example_quantizer_and_delegate/example_operators/TARGETS rename to backends/example/example_operators/TARGETS diff --git a/examples/example_quantizer_and_delegate/example_operators/adaptive_avg_pool2d.py b/backends/example/example_operators/adaptive_avg_pool2d.py similarity index 88% rename from examples/example_quantizer_and_delegate/example_operators/adaptive_avg_pool2d.py rename to backends/example/example_operators/adaptive_avg_pool2d.py index fcd2c6cd347..99042873d65 100644 --- a/examples/example_quantizer_and_delegate/example_operators/adaptive_avg_pool2d.py +++ b/backends/example/example_operators/adaptive_avg_pool2d.py @@ -7,10 +7,8 @@ from dataclasses import dataclass import torch -from executorch.examples.example_quantizer_and_delegate.example_operators.op_base import ( - OpBase, -) -from executorch.examples.example_quantizer_and_delegate.example_operators.utils import ( +from executorch.backends.example.example_operators.op_base import OpBase +from executorch.backends.example.example_operators.utils import ( _annotate_nodes, _nodes_are_annotated, ) diff --git a/examples/example_quantizer_and_delegate/example_operators/add.py b/backends/example/example_operators/add.py similarity index 85% rename from examples/example_quantizer_and_delegate/example_operators/add.py rename to backends/example/example_operators/add.py index a08ca0a8818..aa94093a5cf 100644 --- a/examples/example_quantizer_and_delegate/example_operators/add.py +++ b/backends/example/example_operators/add.py @@ -7,10 +7,8 @@ from dataclasses import dataclass import torch -from executorch.examples.example_quantizer_and_delegate.example_operators.op_base import ( - OpBase, -) -from executorch.examples.example_quantizer_and_delegate.example_operators.utils import ( +from executorch.backends.example.example_operators.op_base import OpBase +from executorch.backends.example.example_operators.utils import ( _annotate_nodes, _nodes_are_annotated, ) diff --git a/examples/example_quantizer_and_delegate/example_operators/conv2d.py b/backends/example/example_operators/conv2d.py similarity index 89% rename from examples/example_quantizer_and_delegate/example_operators/conv2d.py rename to backends/example/example_operators/conv2d.py index 074c41b9de8..8dbf57de4ae 100644 --- a/examples/example_quantizer_and_delegate/example_operators/conv2d.py +++ b/backends/example/example_operators/conv2d.py @@ -7,10 +7,8 @@ from dataclasses import dataclass import torch -from executorch.examples.example_quantizer_and_delegate.example_operators.op_base import ( - OpBase, -) -from executorch.examples.example_quantizer_and_delegate.example_operators.utils import ( +from executorch.backends.example.example_operators.op_base import OpBase +from executorch.backends.example.example_operators.utils import ( _annotate_nodes, _nodes_are_annotated, ) diff --git a/examples/example_quantizer_and_delegate/example_operators/conv_relu.py b/backends/example/example_operators/conv_relu.py similarity index 89% rename from examples/example_quantizer_and_delegate/example_operators/conv_relu.py rename to backends/example/example_operators/conv_relu.py index 7b3fddaee0f..a3a4a570139 100644 --- a/examples/example_quantizer_and_delegate/example_operators/conv_relu.py +++ b/backends/example/example_operators/conv_relu.py @@ -7,10 +7,8 @@ from dataclasses import dataclass import torch -from executorch.examples.example_quantizer_and_delegate.example_operators.op_base import ( - OpBase, -) -from executorch.examples.example_quantizer_and_delegate.example_operators.utils import ( +from executorch.backends.example.example_operators.op_base import OpBase +from executorch.backends.example.example_operators.utils import ( _annotate_nodes, _nodes_are_annotated, ) diff --git a/examples/example_quantizer_and_delegate/example_operators/dropout.py b/backends/example/example_operators/dropout.py similarity index 86% rename from examples/example_quantizer_and_delegate/example_operators/dropout.py rename to backends/example/example_operators/dropout.py index 061e8feeb47..ecaef2b097d 100644 --- a/examples/example_quantizer_and_delegate/example_operators/dropout.py +++ b/backends/example/example_operators/dropout.py @@ -7,10 +7,8 @@ from dataclasses import dataclass import torch -from executorch.examples.example_quantizer_and_delegate.example_operators.op_base import ( - OpBase, -) -from executorch.examples.example_quantizer_and_delegate.example_operators.utils import ( +from executorch.backends.example.example_operators.op_base import OpBase +from executorch.backends.example.example_operators.utils import ( _annotate_nodes, _nodes_are_annotated, ) diff --git a/examples/example_quantizer_and_delegate/example_operators/flatten.py b/backends/example/example_operators/flatten.py similarity index 84% rename from examples/example_quantizer_and_delegate/example_operators/flatten.py rename to backends/example/example_operators/flatten.py index 457fc4493a7..346c9867a5a 100644 --- a/examples/example_quantizer_and_delegate/example_operators/flatten.py +++ b/backends/example/example_operators/flatten.py @@ -7,10 +7,8 @@ from dataclasses import dataclass import torch -from executorch.examples.example_quantizer_and_delegate.example_operators.op_base import ( - OpBase, -) -from executorch.examples.example_quantizer_and_delegate.example_operators.utils import ( +from executorch.backends.example.example_operators.op_base import OpBase +from executorch.backends.example.example_operators.utils import ( _annotate_nodes, _nodes_are_annotated, ) diff --git a/examples/example_quantizer_and_delegate/example_operators/linear.py b/backends/example/example_operators/linear.py similarity index 90% rename from examples/example_quantizer_and_delegate/example_operators/linear.py rename to backends/example/example_operators/linear.py index 7edfe605ca9..d5d36b87f25 100644 --- a/examples/example_quantizer_and_delegate/example_operators/linear.py +++ b/backends/example/example_operators/linear.py @@ -7,10 +7,8 @@ from dataclasses import dataclass import torch -from executorch.examples.example_quantizer_and_delegate.example_operators.op_base import ( - OpBase, -) -from executorch.examples.example_quantizer_and_delegate.example_operators.utils import ( +from executorch.backends.example.example_operators.op_base import OpBase +from executorch.backends.example.example_operators.utils import ( _annotate_nodes, _nodes_are_annotated, ) diff --git a/examples/example_quantizer_and_delegate/example_operators/op_base.py b/backends/example/example_operators/op_base.py similarity index 100% rename from examples/example_quantizer_and_delegate/example_operators/op_base.py rename to backends/example/example_operators/op_base.py diff --git a/examples/example_quantizer_and_delegate/example_operators/ops.py b/backends/example/example_operators/ops.py similarity index 52% rename from examples/example_quantizer_and_delegate/example_operators/ops.py rename to backends/example/example_operators/ops.py index 9c4c9898f2f..56ead1007a2 100644 --- a/examples/example_quantizer_and_delegate/example_operators/ops.py +++ b/backends/example/example_operators/ops.py @@ -6,28 +6,16 @@ from collections import OrderedDict -from executorch.examples.example_quantizer_and_delegate.example_operators.adaptive_avg_pool2d import ( +from executorch.backends.example.example_operators.adaptive_avg_pool2d import ( AdaptiveAvgPool2dNode, ) -from executorch.examples.example_quantizer_and_delegate.example_operators.add import ( - AddNode, -) -from executorch.examples.example_quantizer_and_delegate.example_operators.conv2d import ( - Conv2DNode, -) -from executorch.examples.example_quantizer_and_delegate.example_operators.conv_relu import ( - ConvReluNode, -) -from executorch.examples.example_quantizer_and_delegate.example_operators.dropout import ( - DropOutNode, -) -from executorch.examples.example_quantizer_and_delegate.example_operators.flatten import ( - FlattenNode, -) -from executorch.examples.example_quantizer_and_delegate.example_operators.linear import ( - LinearNode, -) +from executorch.backends.example.example_operators.add import AddNode +from executorch.backends.example.example_operators.conv2d import Conv2DNode +from executorch.backends.example.example_operators.conv_relu import ConvReluNode +from executorch.backends.example.example_operators.dropout import DropOutNode +from executorch.backends.example.example_operators.flatten import FlattenNode +from executorch.backends.example.example_operators.linear import LinearNode # The ordering of this is important as the quantizer will try to match the patterns in this order. # That's why we want to match the fused patterns first and then the non-fused ones. diff --git a/examples/example_quantizer_and_delegate/example_operators/utils.py b/backends/example/example_operators/utils.py similarity index 100% rename from examples/example_quantizer_and_delegate/example_operators/utils.py rename to backends/example/example_operators/utils.py diff --git a/examples/example_quantizer_and_delegate/example_partitioner.py b/backends/example/example_partitioner.py similarity index 94% rename from examples/example_quantizer_and_delegate/example_partitioner.py rename to backends/example/example_partitioner.py index 315578aac8f..11ae7634706 100644 --- a/examples/example_quantizer_and_delegate/example_partitioner.py +++ b/backends/example/example_partitioner.py @@ -7,12 +7,8 @@ from typing import Dict, final import torch -from executorch.examples.example_quantizer_and_delegate.example_backend import ( - TosaBackend, -) -from executorch.examples.example_quantizer_and_delegate.example_operators.ops import ( - module_to_annotator, -) +from executorch.backends.example.example_backend import TosaBackend +from executorch.backends.example.example_operators.ops import module_to_annotator from executorch.exir.backend.canonical_partitioners.pattern_op_partitioner import ( generate_partitions_from_list_of_nodes, ) diff --git a/examples/example_quantizer_and_delegate/example_quantizer.py b/backends/example/example_quantizer.py similarity index 97% rename from examples/example_quantizer_and_delegate/example_quantizer.py rename to backends/example/example_quantizer.py index 7b4a11df49b..7034b9792f3 100644 --- a/examples/example_quantizer_and_delegate/example_quantizer.py +++ b/backends/example/example_quantizer.py @@ -8,9 +8,7 @@ from typing import List import torch -from executorch.examples.example_quantizer_and_delegate.example_operators.ops import ( - module_to_annotator, -) +from executorch.backends.example.example_operators.ops import module_to_annotator from torch import fx from torch.ao.quantization.observer import HistogramObserver, MinMaxObserver from torch.ao.quantization.pt2e.graph_utils import find_sequential_partitions diff --git a/examples/example_quantizer_and_delegate/test_example_delegate.py b/backends/example/test_example_delegate.py similarity index 94% rename from examples/example_quantizer_and_delegate/test_example_delegate.py rename to backends/example/test_example_delegate.py index 9e6ed57f1e6..d6c9f865525 100644 --- a/examples/example_quantizer_and_delegate/test_example_delegate.py +++ b/backends/example/test_example_delegate.py @@ -10,12 +10,8 @@ import torch import torch._export as export from executorch import exir -from executorch.examples.example_quantizer_and_delegate.example_partitioner import ( - ExamplePartitioner, -) -from executorch.examples.example_quantizer_and_delegate.example_quantizer import ( - ExampleQuantizer, -) +from executorch.backends.example.example_partitioner import ExamplePartitioner +from executorch.backends.example.example_quantizer import ExampleQuantizer from executorch.exir.backend.backend_api import to_backend from executorch.exir.backend.canonical_partitioners.duplicate_dequant_node_pass import ( diff --git a/build/cmake_deps.toml b/build/cmake_deps.toml index 114b8613576..35e7e7f08f9 100644 --- a/build/cmake_deps.toml +++ b/build/cmake_deps.toml @@ -13,7 +13,7 @@ excludes = [ [targets.executor_runner] buck_targets = [ - "//examples/executor_runner:executor_runner", + "//examples/portable/executor_runner:executor_runner", ] filters = [ ".cpp$", @@ -104,7 +104,7 @@ deps = [ [targets.xnn_executor_runner] buck_targets = [ - "//examples/backend:xnn_executor_runner", + "//examples/xnnpack:xnn_executor_runner", ] filters = [ ".cpp$", diff --git a/docs/source/getting-started-setup.md b/docs/source/getting-started-setup.md index be920d74e15..10bc45c897b 100644 --- a/docs/source/getting-started-setup.md +++ b/docs/source/getting-started-setup.md @@ -99,17 +99,17 @@ how to do it, we will generate an ExecuTorch program file from an `nn.Module`. You can generate an ExecuTorch program by using a sample script or by using the Python interpreter. -We have created the `export_example.py` script that demonstrates a simple model +We have created the `export.py` script that demonstrates a simple model export to flatbuffer. This script is available -in the [pytorch/executorch](https://github.com/pytorch/executorch/tree/main/examples/export) +in the [pytorch/executorch](https://github.com/pytorch/executorch/tree/main/examples/portable) repository. To generate a sample program, complete the following steps: -1. Run the `export_example.py` script: +1. Run the `export.py` script: ```bash - python3 -m examples.export.export_example --model_name="add" + python3 -m examples.portable.scripts.export --model_name="add" ``` :::{dropdown} Output @@ -187,7 +187,7 @@ Complete the following steps: 3. Build a binary: ```bash - /tmp/buck2 build //examples/executor_runner:executor_runner --show-output + /tmp/buck2 build //examples/portable/executor_runner:executor_runner --show-output ``` :::{dropdown} Output @@ -221,7 +221,7 @@ the `buck run` command to run our program. * To run the `add.pte` program: ```bash - /tmp/buck2 run //examples/executor_runner:executor_runner -- --model_path add.pte + /tmp/buck2 run //examples/portable/executor_runner:executor_runner -- --model_path add.pte ``` :::{dropdown} Sample Output diff --git a/docs/source/runtime-overview.md b/docs/source/runtime-overview.md index 6655f2452fa..2471048740c 100644 --- a/docs/source/runtime-overview.md +++ b/docs/source/runtime-overview.md @@ -157,7 +157,7 @@ However, please note: For more details about the ExecuTorch runtime, please see: * The - [`executor_runner`](https://github.com/pytorch/executorch/blob/main/examples/executor_runner/executor_runner.cpp) + [`executor_runner`](https://github.com/pytorch/executorch/blob/main/examples/portable/executor_runner/executor_runner.cpp) example tool * [Runtime API](runtime-api.md) * [Runtime Build and Cross Compilation](runtime-build-and-cross-compilation.md) diff --git a/docs/website/docs/tutorials/00_setting_up_executorch.md b/docs/website/docs/tutorials/00_setting_up_executorch.md index 5c241b2f92d..5e39a33a174 100644 --- a/docs/website/docs/tutorials/00_setting_up_executorch.md +++ b/docs/website/docs/tutorials/00_setting_up_executorch.md @@ -58,10 +58,10 @@ corresponsing version of the repo. Via python script: ```bash # Creates the file `add.pte` -python3 -m examples.export.export_example --model_name="add" +python3 -m examples.portable.scripts.export --model_name="add" # Creates the delegated program `composite_model.pte`, other options are "whole" and "partition" -python3 -m examples.export.export_and_delegate --option "composite" +python3 -m examples.portable.scripts.export_and_delegate --option "composite" ``` Or via python interpreter: @@ -98,7 +98,7 @@ You may want to copy the `buck2` binary into your `$PATH` so you can run it as ` `executor_runner` is an example wrapper around executorch runtime which includes all the operators and backends ```bash -/tmp/buck2 build //examples/executor_runner:executor_runner --show-output +/tmp/buck2 build //examples/portable/executor_runner:executor_runner --show-output ``` The `--show-output` flag will print the path to the executable if you want to run it directly. @@ -111,11 +111,11 @@ conda install -c conda-forge lld ### Step 3: Run a binary ```bash -# add.pte is the program generated from export_example.py during AOT Setup Step 3 -/tmp/buck2 run //examples/executor_runner:executor_runner -- --model_path add.pte +# add.pte is the program generated from portable/scripts/export.py during AOT Setup Step 3 +/tmp/buck2 run //examples/portable/executor_runner:executor_runner -- --model_path add.pte # To run a delegated model -/tmp/buck2 run //examples/executor_runner:executor_runner -- --model_path composite_model.pte +/tmp/buck2 run //examples/portable/executor_runner:executor_runner -- --model_path composite_model.pte ``` or execute the binary directly from the `--show-output` path shown when building. diff --git a/docs/website/docs/tutorials/profiling.md b/docs/website/docs/tutorials/profiling.md index 34ce7341f80..8726bd90aba 100644 --- a/docs/website/docs/tutorials/profiling.md +++ b/docs/website/docs/tutorials/profiling.md @@ -56,7 +56,7 @@ Here is an example of what a ExecuTorch run + profile + post-procesing workflow This runs the sample program with profiling enabled ```bash cd executorch -buck2 run -c executorch.prof_enabled=true examples/executor_runner:executor_runner -- --model_path add.pte +buck2 run -c executorch.prof_enabled=true examples/portable/executor_runner:executor_runner -- --model_path add.pte ``` Run the post-processing CLI tool that calls into the same API's listed above and prints out the profiling results in a tabulated format in the terminal. diff --git a/docs/website/docs/tutorials/selective_build.md b/docs/website/docs/tutorials/selective_build.md index 16133c09ee2..2ef061dcca4 100644 --- a/docs/website/docs/tutorials/selective_build.md +++ b/docs/website/docs/tutorials/selective_build.md @@ -94,18 +94,18 @@ In addition to that, if we want to select all ops from a kernel library, we can # Select all ops from a yaml file et_operator_library( name = "select_ops_from_yaml", - ops_schema_yaml_target = "//executorch/examples/custom_ops:custom_ops.yaml", + ops_schema_yaml_target = "//executorch/examples/portable/custom_ops:custom_ops.yaml", ) ``` Then in the kernel registration library we can do: ```starlark executorch_generated_lib( name = "select_ops_lib", - custom_ops_yaml_target = "//executorch/examples/custom_ops:custom_ops.yaml", + custom_ops_yaml_target = "//executorch/examples/portable/custom_ops:custom_ops.yaml", functions_yaml_target = "//executorch/kernels/portable:functions.yaml", deps = [ - "//executorch/examples/custom_ops:custom_ops_1", # kernel library - "//executorch/examples/custom_ops:custom_ops_2", # kernel library + "//executorch/examples/portable/custom_ops:custom_ops_1", # kernel library + "//executorch/examples/portable/custom_ops:custom_ops_2", # kernel library "//executorch/kernels/portable:operators", # kernel library ":select_ops_from_yaml", ":select_ops_in_list", @@ -125,7 +125,7 @@ if(SELECT_ALL_OPS) elseif(SELECT_OPS_LIST) gen_selected_ops("" "${SELECT_OPS_LIST}" "") elseif(SELECT_OPS_YAML) - set(_custom_ops_yaml ${EXECUTORCH_ROOT}/examples/custom_ops/custom_ops.yaml) + set(_custom_ops_yaml ${EXECUTORCH_ROOT}/examples/portable/custom_ops/custom_ops.yaml) gen_selected_ops("${_custom_ops_yaml}" "" "") endif() ``` diff --git a/examples/README.md b/examples/README.md index 1e1910e7113..f06dd6de92e 100644 --- a/examples/README.md +++ b/examples/README.md @@ -1,113 +1,56 @@ # Examples -This dir contains scripts and other helper utilities to illustrate an end-to-end workflow to run a torch.nn.module on the ExecuTorch runtime. -It also includes a list of modules, from a simple `Add` to a full model like `MobileNetv2` and `MobileNetv3`, with more to come. + +The series of demos featured in this directory exemplify a broad spectrum of workflows for deploying ML models on edge devices using ExecuTorch. These demos offer practical insights into key processes such as model exporting, quantization, backend delegation, module composition, memory planning, program saving and loading for inference on ExecuTorch runtime. + +ExecuTorch's extensive support spans from simple modules like "Add" to comprehensive models like `MobileNet V3`, `Wav2Letter`, `Llama 2`, and more, showcasing its versatility in enabling the deployment of a wide spectrum of models across various edge AI applications. ## Directory structure ```bash examples -├── apple/ios # Contains iOS demo apps -|── backend # Contains examples for exporting delegate models and running them using custom executor runners -├── custom_ops # Contains examples to register custom operators into PyTorch as well as register its kernels into ExecuTorch runtime -├── example_quantizer_and_delegate # Contains examples to to fully lowered a MobileNetV2 model to the example backend with an example quantizer -├── executor_runner # This is an example C++ wrapper around the ET runtime -├── export # Python helper scripts to illustrate export workflow -├── models # Contains a set of simple to PyTorch models -├── quantization # Contains examples of quantization workflow -├── arm # Contains examples of the Arm TOSA and Ethos-U NPU flows +├── models # Contains a set of popular and representative PyTorch models +├── portable # Contains end-to-end demos for ExecuTorch in portable mode +├── xnnpack # Contains end-to-end ExecuTorch demos with first-party optimization using XNNPack +├── selective_build # Contains demos of selective build for optimizing the binary size of the ExecuTorch runtime +|── arm # Contains demos of the Arm TOSA and Ethos-U NPU flows +|── qualcomm # Contains demos of Qualcomm QNN backend +├── demo-apps # Contains demo apps for Android and iOS +├── third-party # Third-party libraries required for working on the demos └── README.md # This file ``` -## Using the examples - -We will walk through an example model to generate a binary file from a python torch.nn.module -from the `models` dir using scripts from the `export` dir. Then we will run on these binary -model files on the ExecuTorch (ET) runtime. For that we will use `executor_runner`. It is a simple -wrapper for the ExecuTorch runtime to serve as an example. Although simple, it is capable of loading -and executing previously exported binary file(s). - - -1. Following the setup guide in [Setting up ExecuTorch from GitHub](/docs/website/docs/tutorials/00_setting_up_executorch.md) -you should be able to get the basic development environment for ExecuTorch working. - -2. Using the script `export/export_example.py` generate a model binary file by selecting a -model name from the list of available models in the `models` dir. - -```bash -cd executorch # To the top level dir - -# To get a list of example models -python3 -m examples.export.export_example -h - -# To generate a specific pte model -python3 -m examples.export.export_example --model_name="mv2" # for MobileNetv2 - -# This should generate ./mv2.pte file, if successful. -``` +## Using the examples -Use `-h` (or `--help`) to see all the supported models. +A user's journey may commence by exploring the demos located in the [`portable/`](./portable) directory. Here, you will gain insights into the fundamental end-to-end workflow to generate a binary file from a ML model in [portable mode](/docs/website/docs/basics/terminology.md) and run it on the ExecuTorch runtime. -3. Once we have the model binary (pte) file, then let's run it with ExecuTorch runtime using the `executor_runner`. -```bash -buck2 run examples/executor_runner:executor_runner -- --model_path mv2.pte -``` +## Demo of XNNPACK delegation -## Quantization -Here is the [Quantization Flow Docs](/docs/website/docs/tutorials/quantization_flow.md). +The demos in the [`xnnpack/`](./xnnpack) directory provide valuable insights into the process of lowering and executing an ExecuTorch model with built-in performance enhancements. These demos specifically showcase the workflow involving [XNNPACK backend](https://github.com/pytorch/executorch/tree/main/backends/xnnpack) delegation and quantization. -### Generating quantized model -You can generate quantized model with the following command (following example is for mv2, aka MobileNetV2): -```bash -python3 -m examples.quantization.example --model_name "mv2" --so-library "" # for MobileNetv2 -``` +## Demo of Selective Build -Note that the shared library being passed into `example.py` is required to register the out variants of the quantized operators (e.g., `quantized_decomposed::add.out`)into EXIR. To build this library, run the following command if using buck2: -```bash -buck2 build //kernels/quantized:aot_lib --show-output -``` +To understand how to deploy the ExecuTorch runtime with optimization for binary size, explore the demos available in the [`selective_build/`](./selective_build) directory. These demos are specifically designed to illustrate the [Selective Build](/docs/website/docs/tutorials/selective_build.md), offering insights into reducing the binary size while maintaining efficiency. -If on cmake, follow the instructions in `test_quantize.sh` to build it, the default path is `cmake-out/kernels/quantized/libquantized_ops_lib.so`. -This command will print both the original model after capture and quantized model. +## Demo Apps -The flow produces a quantized model that could be lowered through partitioner or the runtime directly. +Explore mobile apps with ExecuTorch models integrated and deployable on Android and iOS in the [`demo-apps/android/`](./demo-apps/android) and [`demo-apps/apple_ios/`](./demo-apps/apple_ios) directories, respectively. -you can also find the valid quantized example models by running: -```bash -buck2 run executorch/examples/quantization:example -- --help -``` +## Demo of ARM -### Running quantized model +TBD -Quantized model can be run via executor_runner, similar to floating point model, via, as shown above: -```bash -buck2 run examples/executor_runner:executor_runner -- --model_path mv2.pte -``` +## Demo of QNN Backend -Note that, running quantized model, requires various quantized/dequantize operators, available in [quantized kernel lib](/kernels/quantized). +Demos of [ExecuTorch QNN Backend](./qualcomm) by Qualcomm -## XNNPACK Backend -Please see [Backend README](backend/README.md) for XNNPACK quantization, export, and run workflow. ## Dependencies -Various models listed in this directory have dependencies on some other packages, e.g. torchvision, torchaudio. -In order to make sure model's listed in examples are importable, e.g. via - -```python -from executorch.examples.models.mobilenet_v3d import MV3Model -m = MV3Model.get_model() -``` -You need to follow the setup guide in [Setting up ExecuTorch from GitHub](/docs/website/docs/tutorials/00_setting_up_executorch.md) to have appropriate packages installed. If you haven't already, install these deps via - -```bash -cd executorch - -bash ./install_requirements.sh -``` +Various models and workflows listed in this directory have dependencies on some other packages. You need to follow the setup guide in [Setting up ExecuTorch from GitHub](/docs/website/docs/tutorials/00_setting_up_executorch.md) to have appropriate packages installed. diff --git a/examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo.xcodeproj/project.pbxproj b/examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo.xcodeproj/project.pbxproj similarity index 100% rename from examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo.xcodeproj/project.pbxproj rename to examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo.xcodeproj/project.pbxproj diff --git a/examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo.xcodeproj/xcshareddata/xcschemes/App.xcscheme b/examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo.xcodeproj/xcshareddata/xcschemes/App.xcscheme similarity index 100% rename from examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo.xcodeproj/xcshareddata/xcschemes/App.xcscheme rename to examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo.xcodeproj/xcshareddata/xcschemes/App.xcscheme diff --git a/examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Resources/Examples/Arctic fox.jpg b/examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Resources/Examples/Arctic fox.jpg similarity index 100% rename from examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Resources/Examples/Arctic fox.jpg rename to examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Resources/Examples/Arctic fox.jpg diff --git a/examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Resources/Examples/Samoyed.jpg b/examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Resources/Examples/Samoyed.jpg similarity index 100% rename from examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Resources/Examples/Samoyed.jpg rename to examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Resources/Examples/Samoyed.jpg diff --git a/examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Resources/Examples/hot pot.jpg b/examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Resources/Examples/hot pot.jpg similarity index 100% rename from examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Resources/Examples/hot pot.jpg rename to examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Resources/Examples/hot pot.jpg diff --git a/examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Sources/App/App.swift b/examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Sources/App/App.swift similarity index 100% rename from examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Sources/App/App.swift rename to examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Sources/App/App.swift diff --git a/examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Sources/App/CameraController.swift b/examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Sources/App/CameraController.swift similarity index 100% rename from examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Sources/App/CameraController.swift rename to examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Sources/App/CameraController.swift diff --git a/examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Sources/App/CameraPreview.swift b/examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Sources/App/CameraPreview.swift similarity index 100% rename from examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Sources/App/CameraPreview.swift rename to examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Sources/App/CameraPreview.swift diff --git a/examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Sources/App/ClassificationController.swift b/examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Sources/App/ClassificationController.swift similarity index 100% rename from examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Sources/App/ClassificationController.swift rename to examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Sources/App/ClassificationController.swift diff --git a/examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Sources/App/ContentView.swift b/examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Sources/App/ContentView.swift similarity index 100% rename from examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Sources/App/ContentView.swift rename to examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Sources/App/ContentView.swift diff --git a/examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Sources/App/CustomViews.swift b/examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Sources/App/CustomViews.swift similarity index 100% rename from examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Sources/App/CustomViews.swift rename to examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Sources/App/CustomViews.swift diff --git a/examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Sources/ImageClassification/ImageClassification.swift b/examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Sources/ImageClassification/ImageClassification.swift similarity index 100% rename from examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Sources/ImageClassification/ImageClassification.swift rename to examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Sources/ImageClassification/ImageClassification.swift diff --git a/examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/MobileNet-Bridging-Header.h b/examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/MobileNet-Bridging-Header.h similarity index 100% rename from examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/MobileNet-Bridging-Header.h rename to examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/MobileNet-Bridging-Header.h diff --git a/examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/MobileNetClassifier.h b/examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/MobileNetClassifier.h similarity index 100% rename from examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/MobileNetClassifier.h rename to examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/MobileNetClassifier.h diff --git a/examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/MobileNetClassifier.mm b/examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/MobileNetClassifier.mm similarity index 100% rename from examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/MobileNetClassifier.mm rename to examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/MobileNetClassifier.mm diff --git a/examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/MobileNetClassifier.swift b/examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/MobileNetClassifier.swift similarity index 100% rename from examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/MobileNetClassifier.swift rename to examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/MobileNetClassifier.swift diff --git a/examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/Module.cpp b/examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/Module.cpp similarity index 100% rename from examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/Module.cpp rename to examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/Module.cpp diff --git a/examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/Module.h b/examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/Module.h similarity index 100% rename from examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/Module.h rename to examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/Module.h diff --git a/examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/Test/MobileNetClassifierTest.swift b/examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/Test/MobileNetClassifierTest.swift similarity index 100% rename from examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/Test/MobileNetClassifierTest.swift rename to examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Sources/MobileNet/Test/MobileNetClassifierTest.swift diff --git a/examples/apple/ios/README.md b/examples/demo-apps/apple_ios/README.md similarity index 73% rename from examples/apple/ios/README.md rename to examples/demo-apps/apple_ios/README.md index ab97f5e0980..103c7703e0d 100644 --- a/examples/apple/ios/README.md +++ b/examples/demo-apps/apple_ios/README.md @@ -101,39 +101,39 @@ cd Release Copy the core libraries: ```bash -mkdir -p ../../examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Frameworks/executorch/ +mkdir -p ../../examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Frameworks/executorch/ cp libexecutorch.a \ libextension_data_loader.a \ - ../../examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Frameworks/executorch/ + ../../examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Frameworks/executorch/ ``` For Portable CPU operators, copy additional libraries: ```bash -mkdir -p ../../examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Frameworks/portable/ +mkdir -p ../../examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Frameworks/portable/ cp libportable_kernels.a \ libportable_ops_lib.a \ - ../../examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Frameworks/portable/ + ../../examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Frameworks/portable/ ``` For CoreML delegate backend, copy additional libraries: ```bash -mkdir -p ../../examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Frameworks/coreml/ +mkdir -p ../../examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Frameworks/coreml/ cp libcoremldelegate.a \ - ../../examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Frameworks/coreml/ + ../../examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Frameworks/coreml/ ``` For XNNPACK delegate backend, copy additional libraries: ```bash -mkdir -p ../../examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Frameworks/xnnpack/ +mkdir -p ../../examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Frameworks/xnnpack/ cp libclog.a \ libcpuinfo.a \ libpthreadpool.a \ libxnnpack_backend.a \ libXNNPACK.a \ - ../../examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Frameworks/xnnpack/ + ../../examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Frameworks/xnnpack/ ``` Then return to the `executorch` directory: @@ -147,8 +147,8 @@ cd ../.. 1. Download MobileNet model labels and bundle them with the app: ```bash -mkdir -p examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Resources/Models/MobileNet/ -curl https://raw.githubusercontent.com/pytorch/hub/master/imagenet_classes.txt -o examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Resources/Models/MobileNet/imagenet_classes.txt +mkdir -p examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Resources/Models/MobileNet/ +curl https://raw.githubusercontent.com/pytorch/hub/master/imagenet_classes.txt -o examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Resources/Models/MobileNet/imagenet_classes.txt ``` 2. Export a MobileNet v3 model backed with XNNPACK delegate and bundle it with @@ -156,10 +156,10 @@ curl https://raw.githubusercontent.com/pytorch/hub/master/imagenet_classes.txt - ```bash export FLATC_EXECUTABLE=$(realpath third-party/flatbuffers/cmake-out/flatc) -python3 -m examples.export.export_example --model_name="mv3" -python3 -m examples.backend.xnnpack_examples --model_name="mv3" --delegate +python3 -m examples.portable.scripts.export --model_name="mv3" +python3 -m examples.xnnpack.aot_compiler --model_name="mv3" --delegate python3 -m examples.export.coreml_export_and_delegate -m "mv3" -cp mv3.pte mv3_coreml.pte mv3_xnnpack_fp32.pte examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo/Resources/Models/MobileNet/ +cp mv3.pte mv3_coreml.pte mv3_xnnpack_fp32.pte examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo/Resources/Models/MobileNet/ ``` ## Final Steps @@ -167,7 +167,7 @@ cp mv3.pte mv3_coreml.pte mv3_xnnpack_fp32.pte examples/apple/ios/ExecuTorchDemo 1. Open the project with Xcode: ```bash -open executorch/examples/apple/ios/ExecuTorchDemo/ExecuTorchDemo.xcodeproj +open executorch/examples/demo-apps/apple_ios/ExecuTorchDemo/ExecuTorchDemo.xcodeproj ``` 2. Set the Header Search Paths for `MobileNetClassifier` target to the directory diff --git a/examples/portable/README.md b/examples/portable/README.md new file mode 100644 index 00000000000..1d296f25259 --- /dev/null +++ b/examples/portable/README.md @@ -0,0 +1,53 @@ +# ExecuTorch in Portable Mode + +This dir contains demos to illustrate an end-to-end workflow of using ExecuTorch in [portable mode](/docs/website/docs/basics/terminology.md). + + +## Directory structure +```bash +examples/portable +├── scripts # Python scripts to illustrate export workflow +│ ├── export.py +│ └── export_and_delegate.py +├── custom_ops # Contains examples to register custom operators into PyTorch as well as register its kernels into ExecuTorch runtime +├── executor_runner # Contains an example C++ wrapper around the ExecuTorch runtime +└── README.md # This file +``` + +## Using portable mode + +We will walk through an example model to generate a `.pte` file in [portable mode](/docs/website/docs/basics/terminology.md) from a python `torch.nn.module` +from the [`models/`](../models) directory using scripts in the `portable/scripts` directory. Then we will run on the `.pte` model on the ExecuTorch runtime. For that we will use `executor_runner`. + + +1. Following the setup guide in [Setting up ExecuTorch from GitHub](/docs/website/docs/tutorials/00_setting_up_executorch.md) +you should be able to get the basic development environment for ExecuTorch working. + +2. Using the script `portable/scripts/export.py` generate a model binary file by selecting a +model name from the list of available models in the `models` dir. + + +```bash +cd executorch # To the top level dir + +# To get a list of example models +python3 -m examples.portable.scripts.export -h + +# To generate a specific pte model +python3 -m examples.portable.scripts.export --model_name="mv2" # for MobileNetv2 + +# This should generate ./mv2.pte file, if successful. +``` + +Use `-h` (or `--help`) to see all the supported models. + +3. Once we have the model binary (`.pte`) file, then let's run it with ExecuTorch runtime using the `executor_runner`. + +```bash +buck2 run examples/portable/executor_runner:executor_runner -- --model_path ./mv2.pte +``` + + +## Custom Operator Registration + +Explore the demos in the [`custom_ops/`](./custom_ops) directory to learn how to register custom operators into ExecuTorch as well as register its kernels into ExecuTorch runtime. diff --git a/examples/portable/TARGETS b/examples/portable/TARGETS new file mode 100644 index 00000000000..89b28d8b28e --- /dev/null +++ b/examples/portable/TARGETS @@ -0,0 +1,17 @@ +load("@fbsource//xplat/executorch/build:runtime_wrapper.bzl", "runtime") + +runtime.python_library( + name = "utils", + srcs = [ + "utils.py", + ], + visibility = [ + "//executorch/examples/...", + ], + deps = [ + "//caffe2:torch", + "//executorch/exir:lib", + "//executorch/exir:tracer", + "//executorch/exir/program:lib", + ], +) diff --git a/examples/export/__init__.py b/examples/portable/__init__.py similarity index 100% rename from examples/export/__init__.py rename to examples/portable/__init__.py diff --git a/examples/custom_ops/CMakeLists.txt b/examples/portable/custom_ops/CMakeLists.txt similarity index 77% rename from examples/custom_ops/CMakeLists.txt rename to examples/portable/custom_ops/CMakeLists.txt index 6c65e92892d..b0c5d0079b3 100644 --- a/examples/custom_ops/CMakeLists.txt +++ b/examples/portable/custom_ops/CMakeLists.txt @@ -4,15 +4,16 @@ # This source code is licensed under the BSD-style license found in the # LICENSE file in the root directory of this source tree. -# Example CMakeLists.txt for registering custom ops into Executorch. In this +# Example CMakeLists.txt for registering custom ops into ExecuTorch. In this # example we have custom ops `my_ops::mul3.out` implemented in C++ in -# `examples/custom_ops/custom_ops_1.cpp`. We also have it registered into EXIR -# in `examples/custom_ops/custom_ops_1.py`. This CMakeLists.txt runs a script to -# generate wrapper code based on the operator-kernel binding defined in -# `examples/custom_ops/custom_ops.yaml`. Then creates a library that contains -# both binding wrapper and the implementation source file. This library can be -# linked into Executorch binary (`executor_runner` in this example) and it is -# ready to run models containing that custom op. +# `examples/portable/custom_ops/custom_ops_1.cpp`. We also have it registered +# into EXIR in `examples/portable/custom_ops/custom_ops_1.py`. This +# CMakeLists.txt runs a script to generate wrapper code based on the operator +# kernel binding defined in `examples/portable/custom_ops/custom_ops.yaml`. +# Then creates a library that contains both binding wrapper and the +# implementation source file. This library can be linked into ExecuTorch binary +# (`executor_runner` in this example) and it is ready to run models containing +# that custom op. cmake_minimum_required(VERSION 3.19) set(CMAKE_EXPORT_COMPILE_COMMANDS ON) @@ -25,7 +26,7 @@ if(NOT PYTHON_EXECUTABLE) endif() # Source root directory for executorch. if(NOT EXECUTORCH_ROOT) - set(EXECUTORCH_ROOT ${CMAKE_CURRENT_SOURCE_DIR}/../..) + set(EXECUTORCH_ROOT ${CMAKE_CURRENT_SOURCE_DIR}/../../..) endif() # Source root directory for pytorch. if(NOT TORCH_ROOT) diff --git a/examples/custom_ops/README.md b/examples/portable/custom_ops/README.md similarity index 95% rename from examples/custom_ops/README.md rename to examples/portable/custom_ops/README.md index 2ccdbfb3e4c..48abfaa31cd 100644 --- a/examples/custom_ops/README.md +++ b/examples/portable/custom_ops/README.md @@ -36,7 +36,7 @@ After the model is exported by EXIR, we need C++ implementations of these custom ``` For how to write these YAML entries, please refer to [`kernels/portable/README.md`](https://github.com/pytorch/executorch/blob/main/kernels/portable/README.md). -Currently we provide 2 build systems that links `my_ops::mul3.out` kernel (written in `custom_ops_1.cpp`) to Executor runtime: buck2 and CMake. Both instructions are listed in `examples/custom_ops/test_custom_ops.sh`(test_buck2_custom_op_1 and test_cmake_custom_op_1). +Currently we provide 2 build systems that links `my_ops::mul3.out` kernel (written in `custom_ops_1.cpp`) to Executor runtime: buck2 and CMake. Both instructions are listed in `examples/portable/custom_ops/test_custom_ops.sh`(test_buck2_custom_op_1 and test_cmake_custom_op_1). ## Selective build diff --git a/examples/backend/TARGETS b/examples/portable/custom_ops/TARGETS similarity index 100% rename from examples/backend/TARGETS rename to examples/portable/custom_ops/TARGETS diff --git a/examples/custom_ops/custom_ops.yaml b/examples/portable/custom_ops/custom_ops.yaml similarity index 100% rename from examples/custom_ops/custom_ops.yaml rename to examples/portable/custom_ops/custom_ops.yaml diff --git a/examples/custom_ops/custom_ops_1.py b/examples/portable/custom_ops/custom_ops_1.py similarity index 94% rename from examples/custom_ops/custom_ops_1.py rename to examples/portable/custom_ops/custom_ops_1.py index b7f6ea76e29..97bdf67491a 100644 --- a/examples/custom_ops/custom_ops_1.py +++ b/examples/portable/custom_ops/custom_ops_1.py @@ -6,7 +6,7 @@ """Example of showcasing registering custom operator through torch library API.""" import torch -from examples.export.export_example import export_to_exec_prog, save_pte_program +from examples.portable.scripts.export import export_to_exec_prog, save_pte_program from executorch.exir import EdgeCompileConfig from torch.library import impl, Library diff --git a/examples/custom_ops/custom_ops_1_out.cpp b/examples/portable/custom_ops/custom_ops_1_out.cpp similarity index 100% rename from examples/custom_ops/custom_ops_1_out.cpp rename to examples/portable/custom_ops/custom_ops_1_out.cpp diff --git a/examples/custom_ops/custom_ops_2.cpp b/examples/portable/custom_ops/custom_ops_2.cpp similarity index 100% rename from examples/custom_ops/custom_ops_2.cpp rename to examples/portable/custom_ops/custom_ops_2.cpp diff --git a/examples/custom_ops/custom_ops_2.py b/examples/portable/custom_ops/custom_ops_2.py similarity index 82% rename from examples/custom_ops/custom_ops_2.py rename to examples/portable/custom_ops/custom_ops_2.py index 5869f039a6d..c9378978342 100644 --- a/examples/custom_ops/custom_ops_2.py +++ b/examples/portable/custom_ops/custom_ops_2.py @@ -11,7 +11,7 @@ import argparse import torch -from examples.export.export_example import export_to_exec_prog, save_pte_program +from examples.portable.scripts.export import export_to_exec_prog, save_pte_program from executorch.exir import EdgeCompileConfig @@ -41,7 +41,7 @@ def main(): "-s", "--so_library", required=True, - help="Provide path to so library. E.g., cmake-out/examples/custom_ops/libcustom_ops_aot_lib.so", + help="Provide path to so library. E.g., cmake-out/examples/portable/custom_ops/libcustom_ops_aot_lib.so", ) args = parser.parse_args() # See if we have custom op my_ops::mul4.out registered @@ -58,8 +58,8 @@ def main(): raise RuntimeError( "Need to specify shared library path to register custom op my_ops::mul4.out into" "EXIR. The required shared library is defined as `custom_ops_aot_lib` in " - "examples/custom_ops/CMakeLists.txt if you are using CMake build, or `custom_ops_aot_lib_2` in " - "examples/custom_ops/targets.bzl for buck2. One example path would be cmake-out/examples/custom_ops/" + "examples/portable/custom_ops/CMakeLists.txt if you are using CMake build, or `custom_ops_aot_lib_2` in " + "examples/portable/custom_ops/targets.bzl for buck2. One example path would be cmake-out/examples/portable/custom_ops/" "libcustom_ops_aot_lib.[so|dylib]." ) print(args.so_library) diff --git a/examples/custom_ops/custom_ops_2_out.cpp b/examples/portable/custom_ops/custom_ops_2_out.cpp similarity index 100% rename from examples/custom_ops/custom_ops_2_out.cpp rename to examples/portable/custom_ops/custom_ops_2_out.cpp diff --git a/examples/custom_ops/targets.bzl b/examples/portable/custom_ops/targets.bzl similarity index 100% rename from examples/custom_ops/targets.bzl rename to examples/portable/custom_ops/targets.bzl diff --git a/examples/custom_ops/test_custom_ops.sh b/examples/portable/custom_ops/test_custom_ops.sh similarity index 80% rename from examples/custom_ops/test_custom_ops.sh rename to examples/portable/custom_ops/test_custom_ops.sh index 10966aa9b80..ea1b86786ee 100644 --- a/examples/custom_ops/test_custom_ops.sh +++ b/examples/portable/custom_ops/test_custom_ops.sh @@ -12,16 +12,16 @@ set -e # shellcheck source=/dev/null -source "$(dirname "${BASH_SOURCE[0]}")/../../.ci/scripts/utils.sh" +source "$(dirname "${BASH_SOURCE[0]}")/../../../.ci/scripts/utils.sh" test_buck2_custom_op_1() { local model_name='custom_ops_1' echo "Exporting ${model_name}.pte" - ${PYTHON_EXECUTABLE} -m "examples.custom_ops.${model_name}" + ${PYTHON_EXECUTABLE} -m "examples.portable.custom_ops.${model_name}" # should save file custom_ops_1.pte echo 'Running executor_runner' - buck2 run //examples/executor_runner:executor_runner \ + buck2 run //examples/portable/executor_runner:executor_runner \ --config=executorch.register_custom_op=1 -- --model_path="./${model_name}.pte" # should give correct result @@ -32,7 +32,7 @@ test_buck2_custom_op_1() { test_cmake_custom_op_1() { local model_name='custom_ops_1' echo "Exporting ${model_name}.pte" - ${PYTHON_EXECUTABLE} -m "examples.custom_ops.${model_name}" + ${PYTHON_EXECUTABLE} -m "examples.portable.custom_ops.${model_name}" # should save file custom_ops_1.pte (rm -rf cmake-out \ && mkdir cmake-out \ @@ -52,13 +52,13 @@ test_buck2_custom_op_2() { local model_name='custom_ops_2' echo 'Building custom ops shared library' - SO_LIB=$(buck2 build //examples/custom_ops:custom_ops_aot_lib_2 --show-output | grep "buck-out" | cut -d" " -f2) + SO_LIB=$(buck2 build //examples/portable/custom_ops:custom_ops_aot_lib_2 --show-output | grep "buck-out" | cut -d" " -f2) echo "Exporting ${model_name}.pte" - ${PYTHON_EXECUTABLE} -m "examples.custom_ops.${model_name}" --so_library="$SO_LIB" + ${PYTHON_EXECUTABLE} -m "examples.portable.custom_ops.${model_name}" --so_library="$SO_LIB" # should save file custom_ops_2.pte - buck2 run //examples/executor_runner:executor_runner \ + buck2 run //examples/portable/executor_runner:executor_runner \ --config=executorch.register_custom_op=2 -- --model_path="./${model_name}.pte" # should give correct result echo "Removing ${model_name}.pte" @@ -98,7 +98,7 @@ test_cmake_custom_op_2() { EXT=$(get_shared_lib_ext) echo "Exporting ${model_name}.pte" - ${PYTHON_EXECUTABLE} -m "examples.custom_ops.${model_name}" --so_library="cmake-out/examples/custom_ops/libcustom_ops_aot_lib$EXT" + ${PYTHON_EXECUTABLE} -m "examples.portable.custom_ops.${model_name}" --so_library="cmake-out/examples/portable/custom_ops/libcustom_ops_aot_lib$EXT" # should save file custom_ops_2.pte echo 'Running executor_runner' diff --git a/examples/custom_ops/TARGETS b/examples/portable/executor_runner/TARGETS similarity index 100% rename from examples/custom_ops/TARGETS rename to examples/portable/executor_runner/TARGETS diff --git a/examples/executor_runner/executor_runner.cpp b/examples/portable/executor_runner/executor_runner.cpp similarity index 100% rename from examples/executor_runner/executor_runner.cpp rename to examples/portable/executor_runner/executor_runner.cpp diff --git a/examples/executor_runner/targets.bzl b/examples/portable/executor_runner/targets.bzl similarity index 92% rename from examples/executor_runner/targets.bzl rename to examples/portable/executor_runner/targets.bzl index 7903507337d..ac74df08141 100644 --- a/examples/executor_runner/targets.bzl +++ b/examples/portable/executor_runner/targets.bzl @@ -33,9 +33,9 @@ def define_common_targets(): # Include quantized ops to be able to run quantized model with portable ops custom_ops_lib = ["//executorch/kernels/quantized:generated_lib"] if register_custom_op == "1": - custom_ops_lib.append("//executorch/examples/custom_ops:lib_1") + custom_ops_lib.append("//executorch/examples/portable/custom_ops:lib_1") elif register_custom_op == "2": - custom_ops_lib.append("//executorch/examples/custom_ops:lib_2") + custom_ops_lib.append("//executorch/examples/portable/custom_ops:lib_2") # Test driver for models, uses all portable kernels and a demo backend. This # is intended to have minimal dependencies. If you want a runner that links diff --git a/examples/portable/scripts/TARGETS b/examples/portable/scripts/TARGETS new file mode 100644 index 00000000000..79c84ece504 --- /dev/null +++ b/examples/portable/scripts/TARGETS @@ -0,0 +1,13 @@ +load("@fbsource//xplat/executorch/build:runtime_wrapper.bzl", "runtime") + +runtime.python_binary( + name = "export", + srcs = [ + "export.py", + ], + main_module = "executorch.examples.portable.scripts.export", + deps = [ + "//executorch/examples/models:models", + "//executorch/examples/portable:utils", + ], +) diff --git a/examples/export/export_example.py b/examples/portable/scripts/export.py similarity index 88% rename from examples/export/export_example.py rename to examples/portable/scripts/export.py index 9c2a9d9362e..474bf116497 100644 --- a/examples/export/export_example.py +++ b/examples/portable/scripts/export.py @@ -9,9 +9,9 @@ import argparse import logging -from ..models import MODEL_NAME_TO_MODEL -from ..models.model_factory import EagerModelFactory -from .utils import export_to_exec_prog, save_pte_program +from ...models import MODEL_NAME_TO_MODEL +from ...models.model_factory import EagerModelFactory +from ..utils import export_to_exec_prog, save_pte_program FORMAT = "[%(levelname)s %(asctime)s %(filename)s:%(lineno)s] %(message)s" diff --git a/examples/export/export_and_delegate.py b/examples/portable/scripts/export_and_delegate.py similarity index 98% rename from examples/export/export_and_delegate.py rename to examples/portable/scripts/export_and_delegate.py index 6e4f57becd7..7285f6b7115 100644 --- a/examples/export/export_and_delegate.py +++ b/examples/portable/scripts/export_and_delegate.py @@ -17,10 +17,10 @@ ) from executorch.exir.backend.test.op_partitioner_demo import AddMulPartitionerDemo -from ..models import MODEL_NAME_TO_MODEL -from ..models.model_factory import EagerModelFactory +from ...models import MODEL_NAME_TO_MODEL +from ...models.model_factory import EagerModelFactory -from .utils import export_to_edge +from ..utils import export_to_edge FORMAT = "[%(levelname)s %(asctime)s %(filename)s:%(lineno)s] %(message)s" diff --git a/examples/export/test/TARGETS b/examples/portable/test/TARGETS similarity index 88% rename from examples/export/test/TARGETS rename to examples/portable/test/TARGETS index b64f45d79b5..dd48aa16a25 100644 --- a/examples/export/test/TARGETS +++ b/examples/portable/test/TARGETS @@ -8,8 +8,8 @@ python_unittest( tags = ["long_running"], deps = [ "//caffe2:torch", - "//executorch/examples/export:utils", "//executorch/examples/models:models", + "//executorch/examples/portable:utils", "//executorch/extension/pybindings:portable_lib", # @manual ], ) diff --git a/examples/export/test/test_export.py b/examples/portable/test/test_export.py similarity index 98% rename from examples/export/test/test_export.py rename to examples/portable/test/test_export.py index 14c049d4f77..062a04987e7 100644 --- a/examples/export/test/test_export.py +++ b/examples/portable/test/test_export.py @@ -10,11 +10,11 @@ import torch import torch._export as export - -from executorch.examples.export.utils import export_to_edge from executorch.examples.models import MODEL_NAME_TO_MODEL from executorch.examples.models.model_factory import EagerModelFactory +from executorch.examples.portable.utils import export_to_edge + from executorch.extension.pybindings.portable_lib import ( # @manual _load_for_executorch_from_buffer, ) diff --git a/examples/export/utils.py b/examples/portable/utils.py similarity index 100% rename from examples/export/utils.py rename to examples/portable/utils.py diff --git a/examples/recipes/xnnpack_optimization/TARGETS b/examples/recipes/xnnpack_optimization/TARGETS deleted file mode 100644 index 8b25df013bf..00000000000 --- a/examples/recipes/xnnpack_optimization/TARGETS +++ /dev/null @@ -1,12 +0,0 @@ -load("@fbcode_macros//build_defs:python_library.bzl", "python_library") - -python_library( - name = "models", - srcs = [ - "__init__.py", - "models.py", - ], - deps = [ - "//executorch/examples/models:models", # @manual - ], -) diff --git a/examples/recipes/xnnpack_optimization/__init__.py b/examples/recipes/xnnpack_optimization/__init__.py deleted file mode 100644 index 95a9180bd55..00000000000 --- a/examples/recipes/xnnpack_optimization/__init__.py +++ /dev/null @@ -1,9 +0,0 @@ -# Copyright (c) Meta Platforms, Inc. and affiliates. -# All rights reserved. -# -# This source code is licensed under the BSD-style license found in the -# LICENSE file in the root directory of this source tree. - -from .models import MODEL_NAME_TO_OPTIONS - -__all__ = [MODEL_NAME_TO_OPTIONS] diff --git a/examples/recipes/xnnpack_optimization/models.py b/examples/recipes/xnnpack_optimization/models.py deleted file mode 100644 index 60bd3568dcc..00000000000 --- a/examples/recipes/xnnpack_optimization/models.py +++ /dev/null @@ -1,31 +0,0 @@ -# Copyright (c) Meta Platforms, Inc. and affiliates. -# All rights reserved. -# -# This source code is licensed under the BSD-style license found in the -# LICENSE file in the root directory of this source tree. - -from dataclasses import dataclass - - -@dataclass -class OptimizationOptions(object): - quantization: bool - xnnpack_delegation: bool - - -MODEL_NAME_TO_OPTIONS = { - "linear": OptimizationOptions(True, True), - "add": OptimizationOptions(True, True), - "add_mul": OptimizationOptions(True, True), - "dl3": OptimizationOptions(True, True), - "ic3": OptimizationOptions(True, False), - "ic4": OptimizationOptions(True, False), - "mv2": OptimizationOptions(True, True), - "mv3": OptimizationOptions(True, True), - "resnet18": OptimizationOptions(True, True), - "resnet50": OptimizationOptions(True, True), - "vit": OptimizationOptions(False, True), - "w2l": OptimizationOptions(False, True), - "edsr": OptimizationOptions(True, False), - "mobilebert": OptimizationOptions(True, False), -} diff --git a/examples/sdk/scripts/TARGETS b/examples/sdk/scripts/TARGETS new file mode 100644 index 00000000000..0b93cd0c0ef --- /dev/null +++ b/examples/sdk/scripts/TARGETS @@ -0,0 +1,13 @@ +load("@fbsource//xplat/executorch/build:runtime_wrapper.bzl", "runtime") + +runtime.python_binary( + name = "export_bundled_program", + main_src = "export_bundled_program.py", + deps = [ + "//executorch/bundled_program:config", + "//executorch/bundled_program:core", + "//executorch/bundled_program/serialize:lib", + "//executorch/examples/models:models", + "//executorch/examples/portable:utils", + ], +) diff --git a/examples/export/export_bundled_program.py b/examples/sdk/scripts/export_bundled_program.py similarity index 95% rename from examples/export/export_bundled_program.py rename to examples/sdk/scripts/export_bundled_program.py index 5339fa58d8e..158c0cbc0c0 100644 --- a/examples/export/export_bundled_program.py +++ b/examples/sdk/scripts/export_bundled_program.py @@ -14,10 +14,9 @@ serialize_from_bundled_program_to_flatbuffer, ) -from ..models import MODEL_NAME_TO_MODEL -from ..models.model_factory import EagerModelFactory - -from .utils import export_to_exec_prog, save_pte_program +from ...models import MODEL_NAME_TO_MODEL +from ...models.model_factory import EagerModelFactory +from ...portable.utils import export_to_exec_prog, save_pte_program def save_bundled_program( diff --git a/examples/selective_build/CMakeLists.txt b/examples/selective_build/CMakeLists.txt index 372b4bdba27..6f0d3638b9a 100644 --- a/examples/selective_build/CMakeLists.txt +++ b/examples/selective_build/CMakeLists.txt @@ -28,11 +28,12 @@ include(${EXECUTORCH_ROOT}/build/Codegen.cmake) # set(_kernel_lib) if(EXECUTORCH_SELECT_OPS_YAML) - set(_custom_ops_yaml ${EXECUTORCH_ROOT}/examples/custom_ops/custom_ops.yaml) + set(_custom_ops_yaml + ${EXECUTORCH_ROOT}/examples/portable/custom_ops/custom_ops.yaml) gen_selected_ops("${_custom_ops_yaml}" "" "") set(kernel_sources - ${EXECUTORCH_ROOT}/examples/custom_ops/custom_ops_1_out.cpp - ${EXECUTORCH_ROOT}/examples/custom_ops/custom_ops_2_out.cpp) + ${EXECUTORCH_ROOT}/examples/portable/custom_ops/custom_ops_1_out.cpp + ${EXECUTORCH_ROOT}/examples/portable/custom_ops/custom_ops_2_out.cpp) # # custom_kernels: C++ kernel implementations of custom ops # diff --git a/examples/selective_build/targets.bzl b/examples/selective_build/targets.bzl index 73d3ff81795..573b3048260 100644 --- a/examples/selective_build/targets.bzl +++ b/examples/selective_build/targets.bzl @@ -44,15 +44,15 @@ def define_common_targets(): # Select all ops from a yaml file et_operator_library( name = "select_ops_from_yaml", - ops_schema_yaml_target = "//executorch/examples/custom_ops:custom_ops.yaml", + ops_schema_yaml_target = "//executorch/examples/portable/custom_ops:custom_ops.yaml", ) executorch_generated_lib( name = "select_ops_from_yaml_lib", - custom_ops_yaml_target = "//executorch/examples/custom_ops:custom_ops.yaml", + custom_ops_yaml_target = "//executorch/examples/portable/custom_ops:custom_ops.yaml", deps = [ - "//executorch/examples/custom_ops:custom_ops_1", - "//executorch/examples/custom_ops:custom_ops_2", + "//executorch/examples/portable/custom_ops:custom_ops_1", + "//executorch/examples/portable/custom_ops:custom_ops_2", ":select_ops_from_yaml", ], ) @@ -73,7 +73,7 @@ def define_common_targets(): name = "selective_build_test", srcs = [], deps = [ - "//executorch/examples/executor_runner:executor_runner_lib", + "//executorch/examples/portable/executor_runner:executor_runner_lib", ] + lib, define_static_target = True, **get_oss_build_kwargs() diff --git a/examples/selective_build/test_selective_build.sh b/examples/selective_build/test_selective_build.sh index 95fd0267a89..0e3b1688930 100644 --- a/examples/selective_build/test_selective_build.sh +++ b/examples/selective_build/test_selective_build.sh @@ -17,7 +17,7 @@ source "$(dirname "${BASH_SOURCE[0]}")/../../.ci/scripts/utils.sh" test_buck2_select_all_ops() { echo "Exporting MobilenetV3" - ${PYTHON_EXECUTABLE} -m examples.export.export_example --model_name="mv3" + ${PYTHON_EXECUTABLE} -m examples.portable.scripts.export --model_name="mv3" echo "Running selective build test" $BUCK run //examples/selective_build:selective_build_test \ @@ -29,7 +29,7 @@ test_buck2_select_all_ops() { test_buck2_select_ops_in_list() { echo "Exporting add_mul" - ${PYTHON_EXECUTABLE} -m examples.export.export_example --model_name="add_mul" + ${PYTHON_EXECUTABLE} -m examples.portable.scripts.export --model_name="add_mul" echo "Running selective build test" # set max_kernel_num=17: 14 primops, add, mul @@ -43,7 +43,7 @@ test_buck2_select_ops_in_list() { test_buck2_select_ops_from_yaml() { echo "Exporting custom_op_1" - ${PYTHON_EXECUTABLE} -m examples.custom_ops.custom_ops_1 + ${PYTHON_EXECUTABLE} -m examples.portable.custom_ops.custom_ops_1 echo "Running selective build test" $BUCK run //examples/selective_build:selective_build_test \ @@ -55,7 +55,7 @@ test_buck2_select_ops_from_yaml() { test_cmake_select_all_ops() { echo "Exporting MobilenetV3" - ${PYTHON_EXECUTABLE} -m examples.export.export_example --model_name="mv3" + ${PYTHON_EXECUTABLE} -m examples.portable.scripts.export --model_name="mv3" (rm -rf cmake-out \ && mkdir cmake-out \ @@ -78,7 +78,7 @@ test_cmake_select_all_ops() { test_cmake_select_ops_in_list() { echo "Exporting MobilenetV2" - ${PYTHON_EXECUTABLE} -m examples.export.export_example --model_name="mv2" + ${PYTHON_EXECUTABLE} -m examples.portable.scripts.export --model_name="mv2" # set MAX_KERNEL_NUM=17: 14 primops, add, mul (rm -rf cmake-out \ @@ -106,7 +106,7 @@ aten,aten::clone.out" \ test_cmake_select_ops_in_yaml() { echo "Exporting custom_op_1" - ${PYTHON_EXECUTABLE} -m examples.custom_ops.custom_ops_1 + ${PYTHON_EXECUTABLE} -m examples.portable.custom_ops.custom_ops_1 (rm -rf cmake-out \ && mkdir cmake-out \ diff --git a/examples/backend/README.md b/examples/xnnpack/README.md similarity index 58% rename from examples/backend/README.md rename to examples/xnnpack/README.md index 535565dd3e7..efd3301c1da 100644 --- a/examples/backend/README.md +++ b/examples/xnnpack/README.md @@ -1,38 +1,79 @@ -This README gives some examples on backend-specific model workflow. - # XNNPACK Backend -[XNNPACK](https://github.com/google/XNNPACK) is a library of optimized of neural network inference operators for ARM and x86 platforms. Our delegate lowers models to run using these highly optimized CPU operators. You can try out lowering and running some example models using the following commands: +[XNNPACK](https://github.com/google/XNNPACK) is a library of optimized of neural network inference operators for ARM and x86 platforms. Our delegate lowers models to run using these highly optimized CPU operators. You can try out lowering and running some example models in the demo. + + +## Directory structure +```bash +examples/xnnpack +├── quantization # Scripts to illustrate PyTorch 2.0 quantization workflow with XNNPack quantizer +│ └── example.py +├── aot_compiler.py # The main script to illustrate the full AOT (export, quantization, delegation) workflow with XNNPACK +├── xnn_executor_runner # ExecuTorch runtime with XNNPack +└── README.md # This file +``` + ## XNNPACK delegation-only -The following command will produce an floating-point XNNPACK delegated model `mv2_xnnpack_fp32.pte` that can be run using XNNPACK's operators. It will also print out the lowered graph, showing what parts of the models have been lowered to XNNPACK via `executorch_call_delegate`. +The following command will produce a floating-point XNNPACK delegated model `mv2_xnnpack_fp32.pte` that can be run using XNNPACK's operators. It will also print out the lowered graph, showing what parts of the models have been lowered to XNNPACK via `executorch_call_delegate`. ```bash # For MobileNet V2 -python3 -m examples.backend.xnnpack_examples --model_name="mv2" --delegate +python3 -m examples.xnnpack.aot_compiler --model_name="mv2" --delegate ``` Once we have the model binary (pte) file, then let's run it with ExecuTorch runtime using the `xnn_executor_runner`. ```bash -buck2 run examples/backend:xnn_executor_runner -- --model_path ./mv2_xnnpack_fp32.pte +buck2 run examples/xnnpack:xnn_executor_runner -- --model_path ./mv2_xnnpack_fp32.pte ``` + ## XNNPACK quantization + delegation -The following command will produce an XNNPACK quantized and delegated model `mv2_xnnpack_q8.pte` that can be run using XNNPACK's operators. It will also print out the lowered graph, showing what parts of the models have been lowered to XNNPACK via `executorch_call_delegate`. + +The following command will produce a XNNPACK quantized and delegated model `mv2_xnnpack_q8.pte` that can be run using XNNPACK's operators. It will also print out the lowered graph, showing what parts of the models have been lowered to XNNPACK via `executorch_call_delegate`. ```bash -python3 -m examples.backend.xnnpack_examples --model_name="mv2" --quantize --delegate +python3 -m examples.xnnpack.aot_compiler --model_name "mv2" --quantize --delegate ``` Once we have the model binary (pte) file, then let's run it with ExecuTorch runtime using the `xnn_executor_runner`. ```bash -buck2 run examples/backend:xnn_executor_runner -- --model_path ./mv2_xnnpack_q8.pte +buck2 run examples/xnnpack:xnn_executor_runner -- --model_path ./mv2_xnnpack_q8.pte +``` + + +## XNNPACK quantization +Learn the generic PyTorch 2.0 quantization workflow in the [Quantization Flow Docs](/docs/website/docs/tutorials/quantization_flow.md). + + +A shared library to register the out variants of the quantized operators (e.g., `quantized_decomposed::add.out`) into EXIR is required. To generate this library, run the following command if using `buck2`: +```bash +buck2 build //kernels/quantized:aot_lib --show-output +``` +Or if on cmake, follow the instructions in `test_quantize.sh` to build it, the default path is `cmake-out/kernels/quantized/libquantized_ops_lib.so`. + +Then you can generate a XNNPACK quantized model with the following command by passing the path to the shared library into the script `quantization/example.py`: +```bash +python3 -m examples.xnnpack.quantization.example --model_name "mv2" --so-library "" # for MobileNetv2 + +# This should generate ./mv2_quantized.pte file, if successful. +``` +You can find more valid quantized example models by running: +```bash +buck2 run executorch/examples/quantization:example -- -h +``` + +A quantized model can be run via `executor_runner`: +```bash +buck2 run examples/portable/executor_runner:executor_runner -- --model_path ./mv2_quantized.pte ``` +Please note that running a quantized model will require the presence of various quantized/dequantize operators in the [quantized kernel lib](../../kernels/quantized). + -## XNNPACK performance gain +## XNNPACK performance ### Overview @@ -40,14 +81,14 @@ We tested the performance for MobileNet V2 and MobileNet V3 on Linux x86 and Mac For each model, we export three variations: portable (without any optimization), xnnpack fp32 (exported for XNNPACK delegation without quantization), xnnpack q8 (exported for XNNPACK delegation with qint8 delegation). -We build the benchmarking binary (will be released in the near future, but it is similar to `examples/backend:xnn_executor_runner`). Benchmarking binary, by default, runs 10 iterations of warmup and 50 iterations of benchmarking. Number reported here are average measured latency, in ms, across 50 runs. The first iteration is slower due to warm up, and the performance is is stable on subsequent iterations, so we also report the execution time for the first iteration for reference. Below is the model execution time for first iteration and subsequent iterations (average after warmup), in milliseconds. We use a single thread to test the models. Details about the methodology and repro steps are below the tables. +We build the benchmarking binary (will be released in the near future, but it is similar to `examples/xnnpack:xnn_executor_runner`). Benchmarking binary, by default, runs 10 iterations of warmup and 50 iterations of benchmarking. Number reported here are average measured latency, in ms, across 50 runs. The first iteration is slower due to warm up, and the performance is is stable on subsequent iterations, so we also report the execution time for the first iteration for reference. Below is the model execution time for first iteration and subsequent iterations (average after warmup), in milliseconds. We use a single thread to test the models. Details about the methodology and repro steps are below the tables. ### Methodology -Models are exported with the steps above for XNNPACK delegation, and with `examples/export:export_example` for portable backend without any optimization. Then use `//examples/backend:xnn_executor_runner` with profiler (command listed below); or in the future, use the runtime in `//sdk/runners:executor_runner` since it gives more options such as number of iterations after build rules for OSS is added. +Models are exported with the steps above for XNNPACK delegation, and with `//examples/portable/scripts:export` for portable backend without any optimization. Then use `//examples/xnnpack:xnn_executor_runner` with profiler (command listed below); or in the future, use the runtime in `//sdk/runners:executor_runner` since it gives more options such as number of iterations after build rules for OSS is added. ``` -buck run -c executorch.prof_enabled=true -c executorch.prof_buf_size=8096 -c executorch.num_prof_blocks=61 //examples/backend:xnn_executor_runner -- --model_path mv3.pte +buck run -c executorch.prof_enabled=true -c executorch.prof_buf_size=8096 -c executorch.num_prof_blocks=61 //examples/xnnpack:xnn_executor_runner -- --model_path mv3.pte ``` A rough number of execution time can be obtained via the log timestamp. The profiler result can be analyzed with `profiler:profiler_results_cli`. diff --git a/examples/executor_runner/TARGETS b/examples/xnnpack/TARGETS similarity index 100% rename from examples/executor_runner/TARGETS rename to examples/xnnpack/TARGETS diff --git a/examples/xnnpack/__init__.py b/examples/xnnpack/__init__.py new file mode 100644 index 00000000000..0878044a816 --- /dev/null +++ b/examples/xnnpack/__init__.py @@ -0,0 +1,34 @@ +# Copyright (c) Meta Platforms, Inc. and affiliates. +# All rights reserved. +# +# This source code is licensed under the BSD-style license found in the +# LICENSE file in the root directory of this source tree. + +from dataclasses import dataclass + + +@dataclass +class XNNPackOptions(object): + quantization: bool + delegation: bool + + +MODEL_NAME_TO_OPTIONS = { + "linear": XNNPackOptions(True, True), + "add": XNNPackOptions(True, True), + "add_mul": XNNPackOptions(True, True), + "dl3": XNNPackOptions(True, True), + "ic3": XNNPackOptions(True, False), + "ic4": XNNPackOptions(True, False), + "mv2": XNNPackOptions(True, True), + "mv3": XNNPackOptions(True, True), + "resnet18": XNNPackOptions(True, True), + "resnet50": XNNPackOptions(True, True), + "vit": XNNPackOptions(False, True), + "w2l": XNNPackOptions(False, True), + "edsr": XNNPackOptions(True, False), + "mobilebert": XNNPackOptions(True, False), +} + + +__all__ = [MODEL_NAME_TO_OPTIONS] diff --git a/examples/backend/xnnpack_examples.py b/examples/xnnpack/aot_compiler.py similarity index 94% rename from examples/backend/xnnpack_examples.py rename to examples/xnnpack/aot_compiler.py index 537072dfff0..318c76eea4a 100644 --- a/examples/backend/xnnpack_examples.py +++ b/examples/xnnpack/aot_compiler.py @@ -15,12 +15,11 @@ from executorch.exir import EdgeCompileConfig from executorch.exir.backend.backend_api import to_backend -from ..export.utils import export_to_edge, save_pte_program - from ..models import MODEL_NAME_TO_MODEL from ..models.model_factory import EagerModelFactory -from ..quantization.utils import quantize -from ..recipes.xnnpack_optimization import MODEL_NAME_TO_OPTIONS +from ..portable.utils import export_to_edge, save_pte_program +from . import MODEL_NAME_TO_OPTIONS +from .quantization.utils import quantize FORMAT = "[%(levelname)s %(asctime)s %(filename)s:%(lineno)s] %(message)s" diff --git a/examples/xnnpack/quantization/TARGETS b/examples/xnnpack/quantization/TARGETS new file mode 100644 index 00000000000..7dea380bc25 --- /dev/null +++ b/examples/xnnpack/quantization/TARGETS @@ -0,0 +1,26 @@ +load("@fbsource//xplat/executorch/build:runtime_wrapper.bzl", "runtime") + +runtime.python_binary( + name = "example", + main_src = "example.py", + preload_deps = ["//executorch/kernels/quantized:aot_lib"], + deps = [ + ":quant_utils", + "//caffe2:torch", + "//executorch/examples/portable:utils", + "//executorch/examples/xnnpack:models", + ], +) + +runtime.python_library( + name = "quant_utils", + srcs = [ + "utils.py", + ], + visibility = [ + "//executorch/examples/...", + ], + deps = [ + "//caffe2:torch", + ], +) diff --git a/examples/quantization/example.py b/examples/xnnpack/quantization/example.py similarity index 97% rename from examples/quantization/example.py rename to examples/xnnpack/quantization/example.py index e38b9814d22..2d231660372 100644 --- a/examples/quantization/example.py +++ b/examples/xnnpack/quantization/example.py @@ -28,11 +28,11 @@ XNNPACKQuantizer, ) -from ..export.utils import export_to_edge, save_pte_program -from ..models import MODEL_NAME_TO_MODEL -from ..models.model_factory import EagerModelFactory -from ..recipes.xnnpack_optimization import MODEL_NAME_TO_OPTIONS +from ...models import MODEL_NAME_TO_MODEL +from ...models.model_factory import EagerModelFactory +from ...portable.utils import export_to_edge, save_pte_program +from .. import MODEL_NAME_TO_OPTIONS from .utils import quantize diff --git a/examples/quantization/test_quantize.sh b/examples/xnnpack/quantization/test_quantize.sh similarity index 85% rename from examples/quantization/test_quantize.sh rename to examples/xnnpack/quantization/test_quantize.sh index 488c2632433..c2c311c0fb4 100644 --- a/examples/quantization/test_quantize.sh +++ b/examples/xnnpack/quantization/test_quantize.sh @@ -10,7 +10,7 @@ set -e # shellcheck source=/dev/null -source "$(dirname "${BASH_SOURCE[0]}")/../../.ci/scripts/utils.sh" +source "$(dirname "${BASH_SOURCE[0]}")/../../../.ci/scripts/utils.sh" get_shared_lib_ext() { UNAME=$(uname) @@ -32,10 +32,10 @@ test_buck2_quantization() { SO_LIB=$($BUCK build //kernels/quantized:aot_lib --show-output | grep "buck-out" | cut -d" " -f2) echo "Run example.py" - ${PYTHON_EXECUTABLE} -m "examples.quantization.example" --so_library="$SO_LIB" --model_name="$1" + ${PYTHON_EXECUTABLE} -m "examples.xnnpack.quantization.example" --so_library="$SO_LIB" --model_name="$1" echo 'Running executor_runner' - $BUCK run //examples/executor_runner:executor_runner -- --model_path="./${1}_quantized.pte" + $BUCK run //examples/portable/executor_runner:executor_runner -- --model_path="./${1}_quantized.pte" # should give correct result echo "Removing ${1}_quantized.pte" @@ -62,7 +62,7 @@ test_cmake_quantization() { SO_LIB="cmake-out/kernels/quantized/libquantized_ops_aot_lib$EXT" echo "Run example.py, shared library $SO_LIB" - ${PYTHON_EXECUTABLE} -m "examples.quantization.example" --so_library="$SO_LIB" --model_name="$1" + ${PYTHON_EXECUTABLE} -m "examples.xnnpack.quantization.example" --so_library="$SO_LIB" --model_name="$1" echo 'Running executor_runner' cmake-out/executor_runner --model_path="./${1}_quantized.pte" diff --git a/examples/quantization/utils.py b/examples/xnnpack/quantization/utils.py similarity index 100% rename from examples/quantization/utils.py rename to examples/xnnpack/quantization/utils.py diff --git a/examples/backend/targets.bzl b/examples/xnnpack/targets.bzl similarity index 58% rename from examples/backend/targets.bzl rename to examples/xnnpack/targets.bzl index e923e1d5d77..4c2caa96d0a 100644 --- a/examples/backend/targets.bzl +++ b/examples/xnnpack/targets.bzl @@ -7,35 +7,47 @@ def define_common_targets(): TARGETS and BUCK files that call this function. """ - runtime.python_binary( - name = "xnnpack_examples", - main_module = "executorch.examples.backend.xnnpack_examples", + runtime.python_library( + name = "models", + srcs = [ + "__init__.py", + ], + visibility = [ + "//executorch/examples/xnnpack/...", + ], deps = [ - ":xnnpack_examples_lib", + "//executorch/examples/models:models", # @manual ], ) runtime.python_library( - name = "xnnpack_examples_lib", + name = "xnnpack_aot_lib", srcs = [ - "xnnpack_examples.py", + "aot_compiler.py", ], deps = [ + ":models", "//executorch/backends/xnnpack/partition:xnnpack_partitioner", - "//executorch/examples/recipes/xnnpack_optimization:models", - "//executorch/examples/quantization:quant_utils", - "//executorch/examples/export:utils", + "//executorch/examples/portable:utils", + "//executorch/examples/xnnpack/quantization:quant_utils", "//executorch/exir:lib", "//executorch/exir/backend:backend_api", ], ) + runtime.python_binary( + name = "aot_compiler", + main_module = "executorch.examples.xnnpack.aot_compiler", + deps = [ + ":xnnpack_aot_lib", + ], + ) + # executor_runner for XNNPACK Backend and portable kernels. runtime.cxx_binary( name = "xnn_executor_runner", - srcs = [], deps = [ - "//executorch/examples/executor_runner:executor_runner_lib", + "//executorch/examples/portable/executor_runner:executor_runner_lib", "//executorch/backends/xnnpack:xnnpack_backend", "//executorch/kernels/portable:generated_lib_all_ops", ],