Arm backend: Add pypi package install option for VGF backend

[mansnils]

New setup option is added: --install-mlsdk-deps-with-pip
For Linux/Windows x86 machines, PyPi packages of MLSDK repository for VGF backend may be used.
This will eventually be the default. Reason it is not yet default is because of a limitation of model-converter to handle large models. The new option will decrease setup time, which can enable VGF backend testing in github.

Co-authored-by: Per Held <per.held@arm.com>, Ryan O'Shea <ryan.oshea3@arm.com>

cc @freddan80 @per @zingo @oscarandersson8218 @digantdesai

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/15551

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 Cancelled Job, 5 Unrelated Failures

As of commit 3a7584f with merge base dcc9a91 ():

CANCELLED JOB - The following job was cancelled. Please retry:

• trunk / test-llama-runner-mac (fp32, xnnpack+custom+quantize_kv) / macos-job (gh)
  ##[error]The operation was canceled.

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, cmake, linux.arm64.2xlarge, execut... / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• trunk / test-huggingface-transformers-xnnpack (smollm3-3b|xnnpack|--quantize) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• trunk / test-models-linux-aarch64 (softmax, portable, linux.arm64.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• trunk / unittest-release / linux / linux-job (gh) (trunk failure)
  [ FAILED ] WavLoaderTest.LoadAudioDataRejectsUnsupportedFormat
• trunk / unittest-release / macos / macos-job (gh) (trunk failure)
  [ FAILED ] WavLoaderTest.LoadAudioDataRejectsUnsupportedFormat

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[zingo]

Hi @SS-JIA this is a PR you might want to test internally if you tests VGF backend.

[zingo]

@SS-JIA Some background: This is needed to be able to add test for VGF backend, currently installing and building VGF tools from source take too long on GitHub to be useful (e.g. jobs times out), but with this we can install a pip packages for most of it and we hope it will be fast enough (like minutes or even seconds) to be able to start adding unit/trunk tests and maybe even the new backend suit tests for VGF. :)

[zingo]

Failed tests are unrelated

[SS-JIA]

@mansnils @zingo Unfortunately seems like I was wrong, this PR did break some internal tests. Would you guys mind creating the PR once more, and I'll import + see what needs to be done to fix the failures!

[zingo]

Go ahead we have not merged the followup PRs that start using this yet so it will probably not be that messy, e.g. only need to rever this so get stuff back to a working state and we reinvestigte, sorry it broke stuff.

[zingo]

Thanks a lot @metascroy for adding the internal fails to the revert PR, with a bit of luck it gives some help resloving this.
I really like it.

This is causing internal test failures:

Error outside of ExecuTorch: ModuleNotFoundError: No module named 'executorch.backends.arm.vgf.model_converter'
Errors inside of ExecuTorch:

buck test '@fbcode//mode/opt' fbcode//executorch/backends/arm/test:cos
buck test '@fbcode//mode/opt' fbcode//executorch/backends/arm/test:custom_partition
buck test '@fbcode//mode/opt' fbcode//executorch/backends/arm/test:debug_hook
buck test '@fbcode//mode/opt' fbcode//executorch/backends/arm/test:decompose_avg_pool2d_pass
buck test '@fbcode//mode/opt' fbcode//executorch/backends/arm/test:decompose_cosine_similarity_pass
buck test '@fbcode//mode/opt' fbcode//executorch/backends/arm/test:decompose_div_pass
buck test '@fbcode//mode/opt' fbcode//executorch/backends/arm/test:decompose_layernorm_pass
buck test '@fbcode//mode/opt' fbcode//executorch/backends/arm/test:decompose_linalg_vector_norm_pass