Expose DirectML provider to python (conflicts resolved from #3359)

[cameronmaske]

Description: Exposed the DirectML (DML) provider to Python.
This is based on #3359 but addresses the conflicts on that branch and include the DirectML.dll in the setup.py.

Motivation and Context
Previously the DML provider was not exposed to python.
Fixes #3358

[ghost]

All CLA requirements met.

[fdwr]

👍

[faxu]

/azp run Linux CPU CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,MacOS CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,MacOS NoContribops CI Pipeline,Linux CPU x64 NoContribops CI Pipeline,Windows CPU CI Pipeline

[faxu]

/azp run orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-mac-ci-pipeline

[azure-pipelines]

Azure Pipelines successfully started running 3 pipeline(s).

[azure-pipelines]

Azure Pipelines successfully started running 9 pipeline(s).

[faxu]

@cameronmaske looks like there are some more conflicts to resolve on this.

[hariharans29]

@cameronmaske @fdwr - does this PR look okay now ? (Resolved conflicts but making sure it makes sense)

[hariharans29]

/azp run Linux CPU CI Pipeline,Linux CPU x64 NoContribops CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,MacOS CI Pipeline,MacOS NoContribops CI Pipeline,Windows CPU CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline

[hariharans29]

/azp run orttraining-linux-ci-pipeline,orttraining-mac-ci-pipeline,orttraining-linux-gpu-ci-pipeline,centos7_cpu,Linux OpenVINO CI Pipeline

[azure-pipelines]

Azure Pipelines successfully started running 5 pipeline(s).

[azure-pipelines]

Azure Pipelines successfully started running 9 pipeline(s).

[hariharans29]

The Windows GPU build are failing. Is it because it isn't able to locate the DirectML.dll for some reason ?

[faxu]

@cameronmaske Can you take a look at the failure issue and also resolve conflicts?

[cameronmaske]

@faxu I’m currently traveling but will try and resolve these issue when I return next week

[cameronmaske]

Is there any way to see the full error output of why the build failed? Clicking Details just gives the error "Build Not Found"

[cameronmaske]

@faxu Issue should be resolved. For context, the DirectML.dll was not being copied over to the /capi directory to be included with the python wheel. I've updated the cmake files to do that now.

[faxu]

/azp run Linux CPU CI Pipeline,Linux CPU x64 NoContribops CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,MacOS CI Pipeline,MacOS NoContribops CI Pipeline,Windows CPU CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline

[faxu]

/azp run orttraining-linux-ci-pipeline,orttraining-mac-ci-pipeline,orttraining-linux-gpu-ci-pipeline,centos7_cpu,Linux OpenVINO CI Pipeline

[azure-pipelines]

Azure Pipelines successfully started running 5 pipeline(s).

[azure-pipelines]

Azure Pipelines successfully started running 9 pipeline(s).

[faxu]

@cameronmaske Still some issues, it seems...

[cameronmaske]

@faux Reading over the test failures, I'd like some guidance from the maintainers about which route to take to address this, as the direction seems quite opinionated.

The error occurring is...

onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Memory pattern must be disabled before registering DMLExecutionProvider

The DML provider does not support enable_mem_pattern = True which is the default behavior. There are two approaches I can think of to address this.

My thoughts are either,

Alter the tests to explicitly disable that option when running with the DMLExecutionProvider. Some special care will need to be taken for when the session options are loaded from the model file.

or

Change the session options to default enable_mem_pattern to False if the DMLExecutionProvider is used. This may be harder to do (I'm very new to the codebase, so guidance is appreciated if this is the route to do).

Maybe there are different approaches, appreciate any thoughts/guidance.

[jywu-msft]

@faux Reading over the test failures, I'd like some guidance from the maintainers about which route to take to address this, as the direction seems quite opinionated.

The error occurring is...

onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Memory pattern must be disabled before registering DMLExecutionProvider

The DML provider does not support enable_mem_pattern = True which is the default behavior. There are two approaches I can think of to address this.

My thoughts are either,

Alter the tests to explicitly disable that option when running with the DMLExecutionProvider. Some special care will need to be taken for when the session options are loaded from the model file.

or

Change the session options to default enable_mem_pattern to False if the DMLExecutionProvider is used. This may be harder to do (I'm very new to the codebase, so guidance is appreciated if this is the route to do).

Maybe there are different approaches, appreciate any thoughts/guidance.

Between those two options, I would lean towards the latter. enable_memory_pattern is an optimization, so if it is incompatible with DMLExecutionProvider, it seems better to make it a no-op, rather than place onus on user to know what optimizations are compatible with which provider.
Defaulting the correct option seems also preferable to changing all the various test code to special case DMLExecutionProvider to instantiate sess_options and set enable_mem_pattern to false.
As for how to default enable_mem_pattern to false, I think we can either remove the error message (and change the setting there). or perhaps better would be to set enable_mem_pattern to false in InferenceSession::ConstructorCommon
sometime after FinalizeSessionOptions()
@pranavsharma , what do you think?

[hariharans29]

This error should go way now, please rebase with master and I can merge this PR. Thanks.

[hariharans29]

/azp run Linux CPU CI Pipeline,Linux CPU x64 NoContribops CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,MacOS CI Pipeline,MacOS NoContribops CI Pipeline,Windows CPU CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline

[hariharans29]

/azp run orttraining-linux-ci-pipeline,orttraining-mac-ci-pipeline,orttraining-linux-gpu-ci-pipeline,centos7_cpu,Linux OpenVINO CI Pipeline

[azure-pipelines]

Azure Pipelines successfully started running 9 pipeline(s).

[azure-pipelines]

Azure Pipelines successfully started running 5 pipeline(s).

[hariharans29]

/azp run Linux CPU CI Pipeline,Linux CPU x64 NoContribops CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,MacOS CI Pipeline,MacOS NoContribops CI Pipeline,Windows CPU CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline

[hariharans29]

/azp run orttraining-linux-ci-pipeline,orttraining-mac-ci-pipeline,orttraining-linux-gpu-ci-pipeline,centos7_cpu,Linux OpenVINO CI Pipeline

[azure-pipelines]

Azure Pipelines successfully started running 9 pipeline(s).

[azure-pipelines]

Azure Pipelines successfully started running 5 pipeline(s).

[cameronmaske]

@hariharans29 Many thanks for the help in getting this merged!