GSoSD Review Discussion Point 5: Should all current kinds of orchestration be supported by the same one orchestration system?

[jerkerdelsing]

To address the late binding and loose coupled SOA properties "orchestrating" who can/should/shall talk to who is important.

Other similar topics are:

• Workflow/choreography
• Plant description
• ...

If we limit the orchestration to who can/should/shall talk to who rules which may be based on policies.
Such rules can be "owned" by

• a consumer
• Plant description microsystem
• workflow/choreography microsystems
• orchestration microsystem
• operator - engineering
• ...

The current hierarchy developed in Arrowhead is

1. Operator - Engineering
2. PlantDescription
3. Orchestration
4. Consumer

For interoperability reasons support for both push and pull distribution of orchestration rules should be supported.

The most important usage scenario I see is a top down definition of orchestration rules possibly with plan A,B,C, ... versions to enable full filament of the desired functionality. The various plans can be created by classical engineering to autonomous AI.

For consumer systems who "know" who to talk to we have two possibilities. Upload such desire to either of Orchestration, PlantDescription or Operator. Or that such consumer will be allowed to perform a 100% bottom up orchestration. Which might open for interesting or serious security, safety, functionality, issues.

[emanuelpalm]

As far as I'm concerned, the service orchestrator helps me determine what services I should consume. If I rather decide that on my own, then I use the service discovery service to discover/search by myself for the services I want.

This division of concerns means that the orchestrator can have one simple single push and/or pull service that looks the same no matter what orchestration strategy is used under the hood. It may even be a simple database of mappings between systems and designated services. Other systems, or an operator, may insert these mappings as part of realizing specific orchestration strategies.

I understand that many interesting and sophisticated orchestration strategies have been proposed as part of the Arrowhead project. I agree with the notion that each of them should be a separate system implementation. I would prefer if all of them could implement the same service, if possible.

[borditamas]

AITIA view

As far as I'm concerned, the service orchestrator helps me determine what services I should consume. If I rather decide that on my own, then I use the service discovery service to discover/search by myself for the services I want.

We think that service discovery should be limited to some fundamental core & support system services for an application system. If arrowhead ensures that a service access details are discovered only via the orchestrator service, then a local cloud operator can utilize additional advantages from the different orchestration strategies (intercloud, QoS, matchmaking etc). See discussion point 9. Of course this could be a configuration decision ("limited or unlimited service discovery") of the cloud operator.

This division of concerns means that the orchestrator can have one simple single push and/or pull service that looks the same no matter what orchestration strategy is used under the hood.

From conceptual point of view it would be a good approach for sure. But on the other hand it would require a quite general input and output data model since a more sophisticated strategy can provide more functionalities and may need more inputs. It could be implemented this way we think, but this approach may be more confusing for the users.

Should all current kinds of orchestration be supported by the same one orchestration system?

We agree that different orchestration strategies should be implemented by different systems.

[jerkerdelsing]

Discussion still ongoing

[DavidRutqvist]

From an architecture perspective:

• Arrowhead should only define a service that given a query responds with what services a system should call.

Then each implementation could define what strategy to use for this. The most simple would be just a table of provider consumer mappings as in the first generation of Arrowhead. A more recent implementation would probably have a more sophisticated approach but the service contract should remain the same. Basically, the architecture does not make any distinction. However, Arrowhead as a framework (set of tools) should provide such a sophisticated implementation.

[jerkerdelsing]

From a foundational viewpoint Orchestration shall provide (pull or push) orchestration rules to consumers.
The rules an Orchestration system can provide will come from e.g. Management tool, PlantDescription, or other sources.
The Orchestration system can store orchestration rules in priority order, Plan A,B,C,....

A to me possible scenario is that most consumers will not have an own understanding of which producer it shall connect to. Thus lifting the orchestration action to a management level using e.g. Management tool, PlantDescription information.

There might be cases (probably rare) where there are strong arguments why the consumer shall know which producer it shall connect to. An interesting question os if and how Arrowhead should be able to track, log and audit consumers acting outside the Orchestration system?

[PerOlofsson-Sinetiq]

Can there be several instances of (various?) Orchestrator systems in an Arrowhead cloud?

[PerOlofsson-Sinetiq]

My opinion concurs with Jerkers idea of having 4 layers of service composition:

• Human
• Plant description (or similar high-level task understanding mechanism)
• Orchestrator (that translates above layer requirements, store and distribute the current setup of consumer-producer connections)
• Consumer (that carries out the connections)

Each of the levels must be able to carry out orchestration by themself for resilience reasons. I also think that all levels must be capable of act in an environment where there are multiple instances that compete in achieving their respective goals.
Ultimately, the rules of the authorisation systems should limit the competing.

[jerkerdelsing]

Each local cloud has it’s own orchestration system. In principle there can be multiple orchestration systems in a local cloud but that will create synchronisation requirement. Thus the recomendation is to have only one Orchestration and Authorisation system in a local cloud.

The orchestration between local clouds is possible using the Gatekeeper and Gateway microsystems in cooperation with ServiceRegistry and Authorisation/(Authentication) microsystems of other local clouds.

[DavidRutqvist]

Maybe that was not the point, but I see no conflict with my statement and what you both stated above :). As long as there is one API/service type used for the different orchestration scenarios I am happy.

[emanuelpalm]

@borditamas

Of course this could be a configuration decision ("limited or unlimited service discovery") of the cloud operator.

I agree with this. Use of the orchestrator should be optional (while it may be practically required for certain kinds of use cases).