Skip to content
This repository was archived by the owner on Nov 17, 2023. It is now read-only.

Improve MaskedSoftmax by oneDNN #20853

Merged
bgawrych merged 5 commits into from
Feb 14, 2022
Merged

Improve MaskedSoftmax by oneDNN #20853

bgawrych merged 5 commits into from
Feb 14, 2022

Conversation

@bgawrych
Copy link
Contributor

@bgawrych bgawrych commented Jan 27, 2022

Description

Utilize few oneDNN primitives to mask input and execute softmax

Accuracy comparision on GluonNLP models (not affected) [C6i.16xlarge]:
image

Performance comaprision (samples/s) [C6i.16xlarge]:
image

BERT Base can archive up to 60% more samples/s with this change

@mxnet-bot
Copy link

mxnet-bot commented Jan 27, 2022

Hey @bgawrych , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

  • To trigger all jobs: @mxnet-bot run ci [all]
  • To trigger specific jobs: @mxnet-bot run ci [job1, job2]

CI supported jobs: [sanity, website, centos-cpu, windows-gpu, miscellaneous, unix-gpu, clang, windows-cpu, edge, unix-cpu, centos-gpu]

Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.

@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress labels Jan 27, 2022
@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Jan 27, 2022
@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Jan 27, 2022
@bgawrych
Copy link
Contributor Author

bgawrych commented Jan 28, 2022

@mxnet-bot run ci [centos-cpu, unix-cpu]

@mxnet-bot
Copy link

mxnet-bot commented Jan 28, 2022

Jenkins CI successfully triggered : [unix-cpu, centos-cpu]

@bgawrych
Copy link
Contributor Author

bgawrych commented Jan 31, 2022

@mxnet-bot run ci [windows-gpu]

@mxnet-bot
Copy link

mxnet-bot commented Jan 31, 2022

Jenkins CI successfully triggered : [windows-gpu]

@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-awaiting-review PR is waiting for code review and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Jan 31, 2022
bartekkuncer
bartekkuncer reviewed Feb 1, 2022
View reviewed changes
@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test and removed pr-awaiting-review PR is waiting for code review pr-awaiting-testing PR is reviewed and waiting CI build and test labels Feb 3, 2022
@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test and removed pr-work-in-progress PR is still work in progress labels Feb 7, 2022
@mseth10 mseth10 added pr-work-in-progress PR is still work in progress and removed pr-awaiting-testing PR is reviewed and waiting CI build and test labels Feb 7, 2022
@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-awaiting-review PR is waiting for code review and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Feb 9, 2022
bartekkuncer
bartekkuncer reviewed Feb 11, 2022
View reviewed changes
@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test and removed pr-awaiting-review PR is waiting for code review labels Feb 11, 2022
bartekkuncer
bartekkuncer approved these changes Feb 11, 2022
View reviewed changes
Copy link
Contributor

@bartekkuncer bartekkuncer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mseth10 mseth10 added pr-work-in-progress PR is still work in progress and removed pr-awaiting-testing PR is reviewed and waiting CI build and test labels Feb 11, 2022
@bgawrych
Copy link
Contributor Author

bgawrych commented Feb 14, 2022

@mxnet-bot run ci [centos-gpu]

@mxnet-bot
Copy link

mxnet-bot commented Feb 14, 2022

Jenkins CI successfully triggered : [centos-gpu]

@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test and removed pr-work-in-progress PR is still work in progress labels Feb 14, 2022
@mseth10 mseth10 added pr-awaiting-review PR is waiting for code review and removed pr-awaiting-testing PR is reviewed and waiting CI build and test labels Feb 14, 2022
@bgawrych bgawrych merged commit 4d915f7 into apache:master Feb 14, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

2 more reviewers

@mozga-intel mozga-intel mozga-intel approved these changes

@bartekkuncer bartekkuncer bartekkuncer approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

pr-awaiting-review PR is waiting for code review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@bgawrych @mxnet-bot @mozga-intel @bartekkuncer @mseth10