Skip to content

Pass flatc executable path via env var for exir serializer.#568

Closed
shoumikhin wants to merge 1 commit intopytorch:mainfrom
shoumikhin:export-D49843232
Closed

Pass flatc executable path via env var for exir serializer.#568
shoumikhin wants to merge 1 commit intopytorch:mainfrom
shoumikhin:export-D49843232

Conversation

@shoumikhin
Copy link
Contributor

@shoumikhin shoumikhin commented Oct 2, 2023 •
edited
Loading

Pre-flight:

  • Clone executorch repo
  • Install the recommended buck2 binary version at /tmp/buck2
  • cd path/to/executorch
  • Sync and update git submodules
  • Make sure you have Python 3.11+ installed (standard on Ventura+) and pip pointing to it
  • Run ./install_requirements.sh to install PyTorch dependencies
# Build flatbuffers compiler

cd third-party/flatbuffers && rm -rf cmake-out && mkdir cmake-out && cd cmake-out

cmake .. && cmake --build . --target flatc

cd ../../..

# Configure executorch

rm -rf cmake-out && mkdir cmake-out && cd cmake-out

cmake .. -G Xcode -DCMAKE_TOOLCHAIN_FILE=../third-party/pytorch/cmake/iOS.cmake -DBUCK2=/tmp/buck2 -DPYTHON_EXECUTABLE=$(which python3) -DFLATC_EXECUTABLE=$(realpath ../third-party/flatbuffers/cmake-out/flatc) -DCMAKE_ARCHIVE_OUTPUT_DIRECTORY=$(pwd) -DEXECUTORCH_BUILD_XNNPACK=ON -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON -DCMAKE_XCODE_ATTRIBUTE_DEVELOPMENT_TEAM=<YOUR_TEAM_ID>  # -DPLATFORM=SIMULATOR

# Build executorch

cmake --build . --config Release

# Copy executorch libs

cd Release

mkdir -p ../../examples/ios_demo_apps/ExecuTorchDemo/ExecuTorchDemo/Frameworks/executorch/

cp libclog.a libcpuinfo.a libexecutorch.a libextension_data_loader.a libportable_kernels.a libportable_ops_lib.a libpthreadpool.a libxnnpack_backend.a libXNNPACK.a ../../examples/ios_demo_apps/ExecuTorchDemo/ExecuTorchDemo/Frameworks/executorch/

# Export a MobileNet v3 model backed with XNNPACK delegate and copy it over to bundle with the app

export FLATC_EXECUTABLE=$(realpath third-party/flatbuffers/cmake-out/flatc)

python3 -m examples.export.export_example --model_name="mv3"
python3 -m examples.backend.xnnpack_examples --model_name="mv3" --delegate

cp mv3.pte mv3_xnnpack_fp32.pte examples/ios_demo_apps/ExecuTorchDemo/ExecuTorchDemo/Resources/Models/MobileNet/

Post-flight:

  • Open executorch/examples/ios_demo_apps/ExecuTorchDemo/ExecuTorchDemo.xcodeproj
  • Set the header search path for MobileNetClassifier target pointing to the dir containing the executorch repo
  • Run the app and tests

Differential Revision: D49843232

@netlify
Copy link

netlify bot commented Oct 2, 2023 •
edited
Loading

✅ Deploy Preview for resplendent-gnome-14e531 canceled.

Name Link
🔨 Latest commit 40af620
🔍 Latest deploy log https://app.netlify.com/sites/resplendent-gnome-14e531/deploys/651b65f113c15d000830bd53

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 2, 2023
@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Oct 2, 2023

This pull request was exported from Phabricator. Differential Revision: D49843232

@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Oct 2, 2023

This pull request was exported from Phabricator. Differential Revision: D49843232

6 similar comments
@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Oct 2, 2023

This pull request was exported from Phabricator. Differential Revision: D49843232

@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Oct 2, 2023

This pull request was exported from Phabricator. Differential Revision: D49843232

@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Oct 2, 2023

This pull request was exported from Phabricator. Differential Revision: D49843232

@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Oct 2, 2023

This pull request was exported from Phabricator. Differential Revision: D49843232

@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Oct 3, 2023

This pull request was exported from Phabricator. Differential Revision: D49843232

@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Oct 3, 2023

This pull request was exported from Phabricator. Differential Revision: D49843232

@shoumikhin @facebook-github-bot
Pass flatc executable path via env var for exir serializer. (#568)
40af620 
Summary: Pull Request resolved: #568

Reviewed By: dbort

Differential Revision: D49843232
@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Oct 3, 2023

This pull request was exported from Phabricator. Differential Revision: D49843232

@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Oct 3, 2023

This pull request has been merged in 99b1539.

tarun292 pushed a commit that referenced this pull request Oct 5, 2023
@shoumikhin @tarun292
Pass flatc executable path via env var for exir serializer. (#568)
da791c0 
Summary: Pull Request resolved: #568

Reviewed By: dbort

Differential Revision: D49843232

fbshipit-source-id: e28768f6e4ec906f3ee794f8e94b1eca712efc54
Gasoonjia pushed a commit that referenced this pull request Jul 30, 2024
@mikekgfb @malfet
perform parallel prefill when possible (#568)
9d10748 
* perform parallel prefill when possible

* typo

* disable hack

* remove print

* remove debug messages which prevent export

* fixes
Gasoonjia pushed a commit that referenced this pull request Jul 30, 2024
@mikekgfb @shoumikhin
@metascroy @malfet @lucylq @jerryzh168 @Jack-Khuu
Quantization, fp acceleration, and testing (#572)
c980472 
* code beautification

* code beautification, move functions together

* make --device fast the default (#515)

* make --device fast the default

* Update iOS.md (#517)

* Update iOS.md

* Update iOS.md

* Pip to pip3 (#504)

* remove macos-12 test

* pip to pip3

* break aoti CI jobs separately (#500)

* init

* fixes

* more fixes

* fixes

* fix

* fix

* bug fix

* add objcopy update

* suppress int8

* undefined variable

---------

Co-authored-by: Michael Gschwind <mikekg@meta.com>

* Support llama3 in chat in run.cpp  (#486)

* refactor chat runner in preparation for llama3

* add sketch for llama3 prompt template and move to returning tokens

* fix tiktoken

* fixes to chat

* add default llama_ver

* Add tests for quantize json, add cuda device specification and precision to cuda.json (#519)

* remove code for no KV Cache path (#527)

* Update ADVANCED-USERS.md (#529)

Update Advanced Users description to reflect changes in the repo since the description was initially created.

* runner-aoti on cuda (#531)

* runner-aoti on cuda

* transfer results back to CPU

* transfer results back to CPU

* runner-aoti on cuda

* Update runner_build.md (#530)

Update description of runner and build process in runner_build.md

* clean up runner code a little (#532)

* clean up runner code a little

* update

* update

* pull out generate loop in chat

* updates

* edit docs

* typo

* move int8 linear class and function into qops.py (#534)

* add dtype tests for runner-aoti + runner-et (#539)

* add dtype tests for runner-aoti + runner-et

* typo

* Quantized embedding (#536)

* move int8 linear class and function into qops.py

* move Quantized Embedding to qops.py

* Move Linear int4 to qops (#537)

* move int8 linear class and function into qops.py

* move Quantized Embedding to qops.py

* move int4 linear to qops

* Revert "add dtype tests for runner-aoti + runner-et (#539)" (#548)

This reverts commit a7a24577a65be67ac9ae4dc05452f35d9c49e5d1.

* fix generate for llama3 (#538)

* fix generate for llama3

* switch more things to C

* remove C++ header

* add delegation visualization instructions (#551)

* Add dtype runner aoti (#552)

* add dtype tests for runner-aoti + runner-et

* typo

* add dtype test runner-aoti

* test sdpa with fp16 (#553)

* test sdpa with fp16

* kv cache fp32

* typo

* update (#560)

* Only support newest versions of lm-eval (#556)

Summary:
remove support for lm-eval 0.3 to reduce the options we have

Test Plan:
CI

Reviewers:

Subscribers:

Tasks:

Tags:

* split cpu eval CI by dtype (#554)

* split cpu eval CI by dtype

* fix

* differentiate names with checks

* keep one name the same as old

* fix

* Removing duplicate HF issue message from README (#559)

Co-authored-by: Michael Gschwind <61328285+mikekgfb@users.noreply.github.com>

* doc updates (#567)

* Add VM-safe MPS check

---------

Co-authored-by: Anthony Shoumikhin <anthony@shoumikh.in>
Co-authored-by: metascroy <161522778+metascroy@users.noreply.github.com>
Co-authored-by: Nikita Shulga <2453524+malfet@users.noreply.github.com>
Co-authored-by: lucylq <lfq@meta.com>
Co-authored-by: Jerry Zhang <jerryzh168@gmail.com>
Co-authored-by: Jack-Khuu <jack.khuu.7@gmail.com>

* add unpacking support (#525)

* add unpacking support

* fix typos and linter

* perform parallel prefill when possible (#568)

* perform parallel prefill when possible

* typo

* disable hack

* remove print

* remove debug messages which prevent export

* fixes

* stream results in generate.py (#571)

* remove logging interfering with export

---------

Co-authored-by: Anthony Shoumikhin <anthony@shoumikh.in>
Co-authored-by: metascroy <161522778+metascroy@users.noreply.github.com>
Co-authored-by: Nikita Shulga <2453524+malfet@users.noreply.github.com>
Co-authored-by: lucylq <lfq@meta.com>
Co-authored-by: Jerry Zhang <jerryzh168@gmail.com>
Co-authored-by: Jack-Khuu <jack.khuu.7@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. fb-exported Merged

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@shoumikhin @facebook-github-bot