Psa fix tensor caching#452
Closed
saurabhkale17 wants to merge 19 commits intoovep-develop-lnl-1.2from
Closed
Conversation
* Implements blob compatibility check for NPU * OVEP catches the NPU driver exception and return failure status * NPU to CPU fallback is disabled when inferencing with blob * Update NPU device exception handling approach * Changes failure status code to exception (std::runtime_error) * Capture all NPU related errors * Throw minimal error message with error type and error code for Release builds * Fix lint issues * Address review comments * Address review comments --------- Co-authored-by: Srirammaswamy <srirammaswamy.s@intel.com>
…PU (#441) * Prototype shared memory allocator on Windows using OV-EP * Partially working allocator. Crashing on tensor destruction. Might have UMD exceptions. Needs further debug. Unknown if values are correct. * Hard code onnx perf to use RT NPU allocator for inputs * Fix allocation lookups coming from different level zero contexts * Page align OV allocation * Allocate input as WC * Only set tensors when they have changed. * Revert "Allocate input as WC" This reverts commit d43219f. * Hard code onnx perf to use RT NPU for outputs * Revert "Hard code onnx perf to use RT NPU for outputs" This reverts commit c1f3b3e. * Hard code onnx perf to use RT NPU for outputs fixed * Fix onnx_perf_test app crash on tensor destroy * refactor: remove redundant ort_shape_to_ovshape lambda function * alocate buffer in NPU visible region from perf test application * remove redundant code * add command line parameter in perf test for using remote tensors * remove redundant code * remove redundant statements * fix crash during inference * remove redundant code * enable backward compatibility of remote tensor feature * Revert "enable backward compatibility of remote tensor feature" This reverts commit 1791b90. * enable backward compatibility of remote tensor feature in OVEP --------- Co-authored-by: Javier E. Martinez <javier.e.martinez@intel.com> Co-authored-by: Eric Crawford <eric.r.crawford@intel.com>
Disable driver caching for NPU when epctx enabled for ov version greater then 2024.3
* fix debug build issue and lint issues * change naming for OVEP NPU specific macro * fix unit tests and lint issues
Generic device memory
Modified Create Options to pass config options to execution Provider
fix psa psr accuracy issue
saurabhkale17
force-pushed
the
ovep-develop-lnl-1.2
branch
from
4d64dc0 to
f9b995c
Compare
Author
|
Duplicate PR of #455 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description