Delegate lifecycle of adapters to reconcilers

[antoineco]

Currently, it is necessary to deploy the splitter and filter adapters ("services") manually, and pass their names to the controller.

routing/config/controller.yaml

Lines 72 to 75 in 2bed9d0

	- name: FILTER_SERVICE
	value: filter-service
	- name: SPLITTER_SERVICE
	value: splitter-service

I might not fully understand why it is that way, and why we even have a central controller, but I think it would but super convenient from an operational perspective if we could let reconcilers deal with the deployment of their own adapters.

We could even retain the multi-tenancy. I think the Zendesk and SNS sources are two good examples of multi-tenant components which only deploy 1 adapter (Ksvc) no matter how many instances of an object are created.

[tzununbekov]

I think I took this mt pattern from the eventing components - adapters are deployed along with the controller. I guess their deployment could be called "manual" but it does not require user/admin intrusion. Does it really change anything if we embed adapter deployments into the controller?

[antoineco]

But it does require a manual creation, unlike usual adapters (sources, targets), right?

My main motivation here is to facilitate the Uber-controller story. If we had one single component to deploy, and then that component would be smart enough to create/update/delete whatever needs to be running or not, things would suddenly become very easy.

Since we already have a lot of experience doing this in sources and targets (especially Zendesk and SNS, which are multi-tenant AND manage their own multi-tenant adapter) I figured it would be a pattern that fits here too.

Refs:

• NewMTGenericServiceReconciler

[tzununbekov]

I'm still not fully understand what means "manual creation". When our CI applies Routing manifests - it creates all resources including controller and adapters. The same approach is used in eventing's MT Broker (deployments) and I perfectly understand why - these adapters should be created "once and forever" with no further updates whatsoever so there is no need to overload the controller's code with this function.
If Uber-controller story requires the controller to reconcile mt adapters, then I guess we have no other options but to make this change.

[tzununbekov]

Our Filter was the first component of Routing and initially, it was a part of the Broker's Attribute filter that's why the whole scheme is similar to MT Broker. I'm not very familiar with the multi-tenancy pattern in the sources, but let me know if the current approach is somehow not suitable for Uber-controller thing.

[antoineco]

To explain the whole thing very shortly, the Uber-controller is not about CI, but about facilitating the deployment of TriggerMesh for customers / PoCs by reducing the amount things to kubectl apply.

The ultimate goal would be to have a single Deployment object to apply, along with the required RBAC roles, and things would just work.

I know things aren't that simple in practice because there might also be webhook deployments+services+secrets involved, but if we could minimize the amount of those "other things", I think it could be a big win. That's what the "operator pattern" was invented for after all.

cc. @sebgoa just to confirm what I'm saying is aligned with what you have in mind ☝️

I'm very familiar with that pattern and I think I would be able to tackle it in a minimum amount of time, so if you want I can assign myself and do it when I'm back. Not an issue at all for me (and also not urgent).

[antoineco]

(Oh, and the statement about the MT controller in eventing is only partially true. For example, the PingSource controller gets scaled to 0 when there is no PingSource, so <opinion>Eventing would have been better off doing it the way we do it I think.</opinion>)

[sebgoa]

cc. @sebgoa just to confirm what I'm saying is aligned with what you have in mind ☝️

yes, that's what I have in mind