Skip to content

Optimize the whisper base demo#94

Merged
fdwr merged 3 commits intomicrosoft:mainfrom
miaobin:whisper_real_time
Sep 23, 2025
Merged

Optimize the whisper base demo#94
fdwr merged 3 commits intomicrosoft:mainfrom
miaobin:whisper_real_time

Conversation

@miaobin
Copy link
Contributor

@miaobin miaobin commented Sep 18, 2025

Optimize the whisper base demo:

  • Initialize all pre-allocated MLTensors and buffer for optimal real-time performance.
  • Set ORT version to test until the fix is included in the latest dev version.
  • Now we can run the demo with IO Binding supporting on the NPU (ORT OpenVINO EP), we have removed the limitation that IO Binding can only run on GPU.

@Honry @fdwr PTAL. cc\ @ibelem Thanks.

Copy link
Collaborator

@fdwr fdwr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor nit, else seems fine to me. Delegating correctness of cached tensors (and whether to relax deviceType for NPU) to Wanming before continuing⏳. Thanks Bin.

Copy link
Contributor

@Honry Honry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@miaobin
Copy link
Contributor Author

miaobin commented Sep 23, 2025

Since the fix is included in the latest dev version. Set up ORT version to dev. Please take another look, thanks a lot!

Copy link
Collaborator

@fdwr fdwr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@fdwr fdwr merged commit c235331 into microsoft:main Sep 23, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants