Skip to content

Qualcomm AI Engine Direct - Suite Operator Test Support Part 2#14848

Merged
cccclai merged 1 commit intopytorch:mainfrom
CodeLinaro:dev1/winskuo/suite_operator_test2
Oct 7, 2025
Merged

Qualcomm AI Engine Direct - Suite Operator Test Support Part 2#14848
cccclai merged 1 commit intopytorch:mainfrom
CodeLinaro:dev1/winskuo/suite_operator_test2

Conversation

@winskuo-quic
Copy link
Collaborator

@winskuo-quic winskuo-quic commented Oct 7, 2025 •
edited by pytorch-bot bot
Loading

Summary

Support following OPs

  • Threshold OP
  • negative dims permute
  • sqrt unit test modified to use desired input rather than random values
  • rsqrt unit test modified to use desired input rather than random values
  • per channel conv3d support

For the sqrt/rsqrt, I believe the sample input for each UT is using rand instead of randn on purpose to prevent negative numbers input, however, if we don't set generate_random_test_inputs=False, then later on it will be using random values consisting of negative numbers, causing nan showing up on output.

If everything works as expected, we should pass 6 more tests, bringing pass rate from 90.7% -> 91.5%

Test plan

UT added

cc @cccclai @shewu-quic @haowhsu-quic @DannyYuyang-quic @cbilgin

@winskuo-quic winskuo-quic requested a review from cccclai as a code owner October 7, 2025 09:48
@pytorch-bot
Copy link

pytorch-bot bot commented Oct 7, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14848

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures

As of commit 5c0e9eb with merge base 9ab5592 (image):

NEW FAILURES - The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 7, 2025
@github-actions
Copy link

github-actions bot commented Oct 7, 2025

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

@winskuo-quic
Copy link
Collaborator Author

winskuo-quic commented Oct 7, 2025

Hi @cccclai,
This PR should support about 5 to 6 more QNN Suite tests. Please have a look. Thanks,

Hi @GregoryComer,
I am working on supporting more QNN operations to pass the suite operator tests. I have made some changes to the suite operator test files. Please let me know if the changes look fine to you. Thanks!

@mergennachin mergennachin added the partner: qualcomm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Qualcomm label Oct 7, 2025
cccclai
cccclai reviewed Oct 7, 2025
View reviewed changes
backends/test/suite/operators/test_rsqrt.py
# 1D tensor
self._test_op(RsqrtModel(), (torch.rand(20) + 0.01,), flow)

self._test_op(
Copy link
Contributor

@cccclai cccclai Oct 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@GregoryComer does the changes in backends/test/suite/operators/* look good to you?

@GregoryComer
Copy link
Member

GregoryComer commented Oct 7, 2025

Hi @cccclai, This PR should support about 5 to 6 more QNN Suite tests. Please have a look. Thanks,

Hi @GregoryComer, I am working on supporting more QNN operations to pass the suite operator tests. I have made some changes to the suite operator test files. Please let me know if the changes look fine to you. Thanks!

Looks good to me. Thanks for catching this and making these updates.

cccclai
cccclai approved these changes Oct 7, 2025
View reviewed changes
Copy link
Contributor

@cccclai cccclai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes under qualcomm/* looks good. @GregoryComer please review the changes in the test/suite

@cccclai cccclai merged commit 0e74a17 into pytorch:main Oct 7, 2025
142 of 146 checks passed
@cccclai
Copy link
Contributor

cccclai commented Oct 7, 2025

@pytorchbot cherry-pick --onto release/1.0 -c regression

pytorchbot pushed a commit that referenced this pull request Oct 7, 2025
@winskuo-quic @pytorchbot
Qualcomm AI Engine Direct - Suite Operator Test Support Part 2 (#14848)
96fae0b 
### Summary

Support following OPs

- Threshold OP
- negative dims permute
- sqrt unit test modified to use desired input rather than random values
- rsqrt unit test modified to use desired input rather than random
values
- per channel conv3d support

For the sqrt/rsqrt, I believe the sample input for each UT is using
`rand` instead of `randn` on purpose to prevent negative numbers input,
however, if we don't set `generate_random_test_inputs=False`, then later
on it will be using random values consisting of negative numbers,
causing `nan` showing up on output.

If everything works as expected, we should pass 6 more tests, bringing
pass rate from **90.7% -> 91.5%**

### Test plan
UT added

cc @cccclai @shewu-quic @haowhsu-quic @DannyYuyang-quic @cbilgin

(cherry picked from commit 0e74a17)
@pytorchbot
Copy link
Collaborator

pytorchbot commented Oct 7, 2025

Cherry picking #14848

The cherry pick PR is at #14869 and it is recommended to link a regression cherry pick PR with an issue. The following tracker issues are updated:

Details for Dev Infra team Raised by workflow job

@pytorchbot pytorchbot mentioned this pull request Oct 7, 2025
Closed
GregoryComer pushed a commit that referenced this pull request Oct 8, 2025
@winskuo-quic @GregoryComer
Qualcomm AI Engine Direct - Suite Operator Test Support Part 2 (#14848)
59617fb 
### Summary

Support following OPs

- Threshold OP
- negative dims permute
- sqrt unit test modified to use desired input rather than random values
- rsqrt unit test modified to use desired input rather than random
values
- per channel conv3d support

For the sqrt/rsqrt, I believe the sample input for each UT is using
`rand` instead of `randn` on purpose to prevent negative numbers input,
however, if we don't set `generate_random_test_inputs=False`, then later
on it will be using random values consisting of negative numbers,
causing `nan` showing up on output.

If everything works as expected, we should pass 6 more tests, bringing
pass rate from **90.7% -> 91.5%**

### Test plan
UT added

cc @cccclai @shewu-quic @haowhsu-quic @DannyYuyang-quic @cbilgin

(cherry picked from commit 0e74a17)
jirioc pushed a commit to nxp-upstream/executorch that referenced this pull request Dec 19, 2025
@winskuo-quic @jirioc
Qualcomm AI Engine Direct - Suite Operator Test Support Part 2 (pytor…
6cbcb41 
…ch#14848)

### Summary

Support following OPs

- Threshold OP
- negative dims permute
- sqrt unit test modified to use desired input rather than random values
- rsqrt unit test modified to use desired input rather than random
values
- per channel conv3d support

For the sqrt/rsqrt, I believe the sample input for each UT is using
`rand` instead of `randn` on purpose to prevent negative numbers input,
however, if we don't set `generate_random_test_inputs=False`, then later
on it will be using random values consisting of negative numbers,
causing `nan` showing up on output.

If everything works as expected, we should pass 6 more tests, bringing
pass rate from **90.7% -> 91.5%**

### Test plan
UT added

cc @cccclai @shewu-quic @haowhsu-quic @DannyYuyang-quic @cbilgin
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@GregoryComer GregoryComer GregoryComer approved these changes

@cccclai cccclai cccclai approved these changes

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. partner: qualcomm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Qualcomm

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@winskuo-quic @GregoryComer @cccclai @pytorchbot @mergennachin