Revert "[cortex_m] Fix linear weight layout: transpose in AOT pass, a…#16910
Revert "[cortex_m] Fix linear weight layout: transpose in AOT pass, a…#16910psiddh merged 1 commit intopytorch:mainfrom
Conversation
…lign meta/ref impl (pytorch#16782)" This reverts commit 06f10b9.
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/16910
Note: Links to docs will display an error until the docs builds have been completed. ⏳ 2 Pending, 1 Unrelated FailureAs of commit 59fdd72 with merge base 5690d26 ( FLAKY - The following job failed but was likely due to flakiness present on trunk:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
bot
added
the
CLA Signed
This PR needs a
|
There was a problem hiding this comment.
Pull request overview
This PR reverts commit 06f10b9 (PR #16782), which fixed the linear weight layout by transposing weights in the AOT pass to align with CMSIS-NN expectations. The revert removes the weight transpose from the AOT compilation pass and reintroduces transpose operations in the Python reference implementation.
Changes:
- Removes weight transpose from
_get_linear_replacementin the AOT pass - Adds weight transpose back to the Python reference implementation at runtime
- Changes meta function to use
weights.shape[0]instead ofweights.shape[1]
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| backends/cortex_m/passes/convert_to_cortex_m_pass.py | Removes AOT weight transpose; passes non-transposed weights to runtime; adds transpose back to kernel_sum computation |
| backends/cortex_m/ops/operators.py | Updates meta function to use shape[0]; adds .T transpose operations in reference implementation |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| args = ( | ||
| node.args[0], | ||
| weights_transposed_node, | ||
| weights, |
There was a problem hiding this comment.
This revert creates an inconsistency between the Python reference implementation and the C++ CMSIS-NN runtime. After this revert, weights are passed to the C++ runtime in PyTorch's [out_features, in_features] format without transposition. However, the C++ runtime at op_quantized_linear.cpp:80 sets filter_dims to {in_feat, 1, 1, out_feat}, indicating CMSIS-NN expects weights in [in_features, out_features] format. The reference implementation compensates by adding .T at runtime (operators.py:389, 399), but the C++ runtime does not perform this transpose, which will cause incorrect results when running on actual hardware.
| ) -> torch.Tensor: | ||
|
|
||
| shape = (*input.shape[:-1], weights.shape[1]) | ||
| shape = (*input.shape[:-1], weights.shape[0]) |
There was a problem hiding this comment.
The meta function now uses weights.shape[0] which assumes weights are in [out_features, in_features] format (PyTorch convention). However, this is inconsistent with the C++ runtime expectation. If weights were properly transposed in the AOT pass (as PR #16782 attempted), this should use weights.shape[1] instead. The current implementation will produce incorrect output shapes when used with the C++ runtime.
| shape = (*input.shape[:-1], weights.shape[0]) | |
| shape = (*input.shape[:-1], weights.shape[1]) |
…lign meta/ref impl (#16782)"
This reverts commit 06f10b9.
Summary
[PLEASE REMOVE] See CONTRIBUTING.md's Pull Requests for ExecuTorch PR guidelines.
[PLEASE REMOVE] If this PR closes an issue, please add a
Fixes #<issue-id>line.[PLEASE REMOVE] If this PR introduces a fix or feature that should be the upcoming release notes, please add a "Release notes: " label. For a list of available release notes labels, check out CONTRIBUTING.md's Pull Requests.
Test plan
[PLEASE REMOVE] How did you test this PR? Please write down any manual commands you used and note down tests that you have written if applicable.