OpenVINO EP v2.0#3585

Merged

jywu-msft merged 286 commits intomicrosoft:masterfrom

intel:openvino-ep-v2

Apr 24, 2020

Contributor

smkarlap commented Apr 18, 2020

Description:
Version update for OpenVINO Execution Provider. Based on the latest OpenVINO release 2020.2 which now includes nGraph stack. Model conversion from ONNX to OpenVINO IR is now done using nGraph ONNX importer component.

Motivation and Context

Supports a larger set of ONNX operators
Supports inference of partial models using sub-graph partitioning
Supports models with symbolic dimensions in shapes.
Uses OpenVINO APIs with Long Term Support (LTS) providing extended support and backward compatibility.
Upto 40% reduction in memory footprint.
Avoids using the python components used by existing version EP thus avoiding expensive python interpreter startup and module conflicts arising from it.

ynimmaga and others added 30 commits

November 14, 2019 14:05


          Added FP16 transformations

85caec4


          Revert "Added CMAKE_BUILD_TYPE to make building dynamic"

f779bb0

This reverts commit d3e17af.


          Added FP16 transformations for FP16 builds

e4af542


          Backend logic cleanup

5380a3f

Cleans the backend(intel_graph.*) code in the following ways:-

1. Minimize global usage: Since all the IR graphs need to be
re-generated on every Infer, it is bad practice to rely on globals
for their saving and usage as there would be multiple readers and
writers to the same global variable leading to incorrect usages or
contentions. This change replaces globals with locals where possible.
 This change also fixes an existing bug with due to
incorrect global usage.

2. Remove all unused functions.

3. Remove all unused headers and prepocessor directives.


          Merge remote-tracking branch 'origin/uep_unsupported_ops' into unifie…

ae4bf5d

…d-ep-dev


          Merge branch 'unified-ep-dev' into manohar/test-runner-bug

d385946


          removed commented out code

290b418


          Merge branch 'manohar/test-runner-bug' into 'unified-ep-dev'

6e52ac7

Backend logic cleanup

See merge request iot-edge/onnxrt-unified-ep/onnxruntime!9


          Disabled default optimization for Intel EP

0ac8812

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>


          Fix missed plugins.xml for python bindings

f22b3a6


          Merge remote-tracking branch 'origin/master' into unified-ep-dev

2a2cab8


          Merge branch 'mt/missed_plugins_xml' into 'unified-ep-dev'

b11a657

Fix missed plugins.xml for python bindings

See merge request iot-edge/onnxrt-unified-ep/onnxruntime!10


          Fixed the build after latest master changes

f4edc5c

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>


          Merge remote-tracking branch 'origin/master' into unified-ep-dev

cf79b4a


          Disabled unsupported ops for accelerators

75dc9d7

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>


          Added some more disabled ops

d748ac6

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>


          Added environment variable to enable debugging

3b4f975

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>


          Added more debug statements

9bb5f9c

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>


          Merge branch 'debug_env' into 'unified-ep-dev'

253aa52

Added environment variable to enable debugging

See merge request iot-edge/onnxrt-unified-ep/onnxruntime!11


          Merge remote-tracking branch 'origin/master' into unified-ep-dev

aae63ae


          Fixed unsupported ops list for GPU and VPU

ac6ef88

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>


          Fixed unsqueeze unit tests

71bea20

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>


          Merge branch 'unsqueeze_fix' into 'unified-ep-dev'

a711e31

Unsqueeze fix

See merge request iot-edge/onnxrt-unified-ep/onnxruntime!12


          Added error message to the status

27b3468

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>


          Overwrite Model proto with shape info from data

feed36b

Overwrites the shape info of Model proto with the shape from
actual input data. Needed for inferring models with Dynamic
shapes.


          Removed print statement and disabled where op

b4fc6f3

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>


          Disabled Reshape with Empty initializer

cb6787c


          Added more debug statements for 1P

7ca6036


          Don't allow 1D inputs with symbol for dimension

388192d


          Merge remote-tracking branch 'origin/master' into unified-ep-dev

suryasidd added 3 commits

April 21, 2020 20:53


          Enabled unnecessarily disabled tests

37c2ae2


          Fixed ngraph protobuf patch

d0dbc38


          Fixed error in protobuf patch

0ec3dc2

jywu-msft reviewed

View reviewed changes

onnxruntime/core/providers/openvino/backend_manager.cc Outdated Show resolved Hide resolved

Contributor

HectorSVC commented Apr 22, 2020

/azp run Linux CPU CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,MacOS CI Pipeline,Win CPU x86 CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,Win CPU x64 NoContribops CI Pipeline,MacOS NoContribops CI Pipeline,Linux CPU x64 NoContribops CI Pipeline,Windows CPU CI Pipeline,Linux OpenVINO CI Pipeline

azure-pipelines bot commented Apr 22, 2020

Azure Pipelines successfully started running 10 pipeline(s).

Member

jywu-msft commented Apr 22, 2020

build failure:
https://dev.azure.com/onnxruntime/onnxruntime/_build/results?buildId=138031&view=logs&j=8bd2c674-eba8-575c-eb57-9ac6de2339b9&t=a733473d-2b4d-5cee-1c4f-4e6c15baf93c&l=10226


          Revert "Use ${OPENVINO_VERSION} to name openvino install directory in…

9fa9c20

… CI builds"

This reverts commit 89e72ad.

Contributor Author

smkarlap commented Apr 22, 2020

The above pipeline failure seems to be due to Inference Engine headers not being found, which is likely due to the OpenVINO paths not being set. The only commit that modified the way paths are set is the recent change that uses the variable OPENVINO_VERSION to set the paths. The docker image built using this changes seems to have all the paths set accurately when tested on my local workstation. However, platform differences between my workstation and the CI machine may be causing this to fail.
I have reverted that commit. Could you please run the pipelines now? If it fails, it will help narrow down the fault.


          Remove unsetting OPTIONAL macro

93f30c5

This is no longer used in recent ONNX update onnx/onnx@da13be2,
so this unset workaround is no longer necessary.

Contributor

HectorSVC commented Apr 22, 2020

/azp run Linux OpenVINO CI Pipeline

azure-pipelines bot commented Apr 22, 2020

Azure Pipelines successfully started running 1 pipeline(s).

smkarlap added 2 commits

April 22, 2020 17:59


          Use a null string default argument for C# API

6fedeab


          Set OpenVINO version yml files and pass to CI Docker builds

c8f24b8

Git Tag info for DLDT as well as install directory are set
using this value.

This reverts commit 9fa9c20.

Member

jywu-msft commented Apr 23, 2020

/azp run Linux OpenVINO CI Pipeline

azure-pipelines bot commented Apr 23, 2020

Azure Pipelines successfully started running 1 pipeline(s).

Member

jywu-msft commented Apr 23, 2020

/azp run Linux CPU CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,MacOS CI Pipeline,Win CPU x86 CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,Win CPU x64 NoContribops CI Pipeline,MacOS NoContribops CI Pipeline,Linux CPU x64 NoContribops CI Pipeline,Windows CPU CI Pipeline

azure-pipelines bot commented Apr 23, 2020

Azure Pipelines successfully started running 9 pipeline(s).


          Documentation: recommendation and instructions for disabling ORT grap…

0ab8ebb

…h optimizations

Member

jywu-msft commented Apr 23, 2020

setup.py has conflict

smkarlap and others added 3 commits

April 23, 2020 14:43


          more doc updates

a92b5bb


          Merge latest updates from branch 'master' into openvino-ep-v2

2e058d6


          Reduced the number of models according to CI time constraints

cac6736

Member

jywu-msft commented Apr 24, 2020

/azp run Linux OpenVINO CI Pipeline

azure-pipelines bot commented Apr 24, 2020

Azure Pipelines successfully started running 1 pipeline(s).

Contributor

HectorSVC commented Apr 24, 2020

/azp run Linux CPU CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,MacOS CI Pipeline,Win CPU x86 CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,Win CPU x64 NoContribops CI Pipeline,MacOS NoContribops CI Pipeline,Linux CPU x64 NoContribops CI Pipeline,Windows CPU CI Pipeline

azure-pipelines bot commented Apr 24, 2020

Azure Pipelines successfully started running 9 pipeline(s).

jywu-msft approved these changes

View reviewed changes

jywu-msft merged commit 6d4f2f5 into microsoft:master

smkarlap deleted the openvino-ep-v2 branch

September 16, 2020 19:56

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet