[Spike] Possiblity to deploy flagd as a grpc sync server

[Kavindu-Dodan]

With the introduction of grpc syncs [1] flagd now has the ability to securely (tls encryption) communicate to a grpc sync provider server implementation.

Given that flagd has built-in sync capability on various sources (file, grpc & k8s), it is worth investigating the possibility to build grpc server version of the flagd. This server build can then be used with OFO, where OFO-controlled side cars get connected to flagd server on top of grpc.

Consider the high-level conceptual architecture,

Tasks

A POC with following topics

• Identify reusable components of flagd
• Local build process (go build flags/ packaging ...)
• Image build process

[1] - #249

[Kavindu-Dodan]

My initial observations are as below,

• We can utilize sync providers as they have a clear contract [1]
• We cannot use the current runtime implementation as it is tightly coupled to the evaluator & notification mechanism
• Current flag store is also tightly coupled to the eventing mechanism. Hence not possible to reuse if we need an intermediate store
• Configurations - need a clear definition of what sync sources we should use for the server deployment.
• Documentation - current documentation is tightly coupled to the evaluation of flags. So if we need a server deployment, we need to put effort to restructure them

With this background, I am leaning toward a dedicated project focusing on server implementation. The simple reason is the limited benefit we will gain by building the server into flagd itself.

cc @beeme1mr @toddbaert @james-milligan @skyerus

[1] - https://github.com/open-feature/flagd/blob/main/pkg/sync/isync.go

[Kavindu-Dodan]

The alternative approach is to avoid internal caching - the store altogether. Then make the "server" focus only on K8s CRDs, resolving permission-related concerns of flagd.

Consider the following,

If flagd-server only acts as a "proxy" to K8s resources, making it the only workload that requires elevated K8s API access and combining that with the "no-store" (no parsing of flag definitions), we should have a simple implementation.

[Kavindu-Dodan]

POC is ready. Updated component relationship diagram is given below,

[Kavindu-Dodan]

Consider the following options from POC onwards,

• Single CRD watcher : This is the POC and the candidate for the first version of the implementation
• Multi-CRD watcher - this can be done as a follow-up on top of the existing implementation
• On-demand CRD watcher - In my view this is a complex task and should be an extended goal if there is a demand for this use case.

[james-milligan]

I have concerns regarding the Single and Multi implementations, in both of these examples the flagd server would need to be aware of all crds on startup, and would need to watch each of them permanently.

In the on-demand implementation a developer could introduce a new CR whilst the server instance is running and sync to it without bouncing the server. This also introduces the opportunity to drop a sync when there are no longer any subscribers for it, reducing load on the api.

[Kavindu-Dodan]

I have concerns regarding the Single and Multi implementations, in both of these examples the flagd server would need to be aware of all crds on startup, and would need to watch each of them permanently.

In the on-demand implementation a developer could introduce a new CR whilst the server instance is running and sync to it without bouncing the server. This also introduces the opportunity to drop a sync when there are no longer any subscribers for it, reducing load on the api.

💯 correct

If we have a requirement to support the dynamic use case, then we should aim for the third solution.