Add support for dynamic workload type

[preetha-intel]

Description

Support to set EPdynamic options in OVEP

Motivation and Context

relate to microsoft#22282

[sfatimar]

Please fix cpp lint issues

[hylandk-movidius]

Hi Preetha -

1] the string comparison needed an explicit cast to std::string for it to work in my testcase.
ORIG - if ((std::find(keys.begin(), keys.end(), "ep.dynamic.workload_type") != keys.end()) &&
(std::find(values.begin(), values.end(),"Efficient") != values.end())) {
backend_manager_->dynamic_workload_type = "EFFICIENT";
}
MOD - if ((std::find(keys.begin(), keys.end(), std::string("ep.dynamic.workload_type")) != keys.end()) &&
(std::find(values.begin(), values.end(), std::string("Efficient")) != values.end())) {
backend_manager_->dynamic_workload_type = "EFFICIENT";
}

2] The requirement from MSFT is that the paramter is not coupled to a inference request. If you recall, they detailed a concern in the original code they provided where they wanted to move from efficient to default for work that had not returned. To do that they moved from changing the parameter in conjuction with an infer request to an asyncronous setting. To do that, we need to set the cmpiled model parameter from the openvin_execution_provider directly (via a pointer to the compiled model for that session). See sample code here - https://github.com/intel-sandbox/AIPPG/tree/main/src/PowerPerformanceGuide/assets/Toolbox/efficient_cpp

[preetha-intel]

Hi Preetha -

1] the string comparison needed an explicit cast to std::string for it to work in my testcase. ORIG - if ((std::find(keys.begin(), keys.end(), "ep.dynamic.workload_type") != keys.end()) && (std::find(values.begin(), values.end(),"Efficient") != values.end())) { backend_manager_->dynamic_workload_type = "EFFICIENT"; } MOD - if ((std::find(keys.begin(), keys.end(), std::string("ep.dynamic.workload_type")) != keys.end()) && (std::find(values.begin(), values.end(), std::string("Efficient")) != values.end())) { backend_manager_->dynamic_workload_type = "EFFICIENT"; }

2] The requirement from MSFT is that the paramter is not coupled to a inference request. If you recall, they detailed a concern in the original code they provided where they wanted to move from efficient to default for work that had not returned. To do that they moved from changing the parameter in conjuction with an infer request to an asyncronous setting. To do that, we need to set the cmpiled model parameter from the openvin_execution_provider directly (via a pointer to the compiled model for that session). See sample code here - https://github.com/intel-sandbox/AIPPG/tree/main/src/PowerPerformanceGuide/assets/Toolbox/efficient_cpp

Hi @hylandk-movidius

Yeah you are right with the string comparison logic. I found it during validation. I have modified it to locate the values independent of the keys location. As they are two separate char arrays an end user may not be assigning values and keys in same order. If we want to restrict I can modify this.

[MayureshV1]

@preetha-intel , Can we set the OV WorkloadType property inside this function directly and not rely for Compute/ Infer to be called which is why MSFT made the API change

Below are 2 lines of code in backendmanager.cc->Compute which we need to call in SetEpDynamicOptions function for both Default and Efficient.
auto ov_execution_network = concrete_backend_->GetOVCompiledModel();
ov_execution_network.set_property({{"WORKLOAD_TYPE", "DEFAULT"}});