Refactor Operator with Multi API design

[mgoerens]

Multi API redesign

Objectives

Break down the reconciliation logic of the Synapse controller.
Allow a more modular approach: Can add or remove bridges individually
Also simplifies / breaks in smaller pieces the integration tests

APIs

Synapse Controller

Watches SYNAPSE CRD:

If Homeserver.ConfigMap
    Copy user-provided ConfigMap
else
    Create New ConfigMap for Synapse with configuration values provided by user in Homeserver.Values

Create Service

If createNewPostgreSQL
    Create PostgresCluster for Synapse
    Edit Synapse ConfigMap to add DB connection info

Create SA, RB, PVC and Deployment for Synapse

Watched HEISENBRIDGE CRD:

If Heisenbridge is deleted:
    Edit ConfigMap for Synapse to REMOVE heisenbridge as an app_service
    Edit Synapse Deployment to REMOVE the mount of the Heisenbridge ConfigMap on /data-heisenbridge

Check if Heisenbridge.Synapse exists
(Wait for Heisenbridge IP to be known ? for Heisenbridge to be up ? ping bridge endpoint ? make some request to Heisenbridge endpoint ?) => wait for state "Waiting for synapse" ?

Edit ConfigMap for Synapse to ADD heisenbridge as an app_service
Edit Synapse Deployment to mount Heisenbridge ConfigMap on /data-heisenbridge

Same logic for further bridges, such as mautrix-signal

Heisenbridge Controller

Watches HEISENBRIDGE CRD:

If Config.ConfigMap
    Copy user-provided ConfigMap
else
    Create New ConfigMap for Heisenbridge with configuration values provided by user in Config.Values

Create Service for Heisenbridge
(Edit Synapse ConfigMap with Synapse IP)

Edit Heisenbridge ConfigMap with Heisenbridge IP and Synapse IP

Create Deployment for Heisenbridge

Watches SYNAPSE CRD:

If synapse deleted
    Delete heisenbridge associated to it, if any

mautrix-signal Controller

Watches MAUTRIX-SIGNAL CRD:

If Config.ConfigMap
    Copy user-provided ConfigMap
else
    Create New ConfigMap for mautrix-signal with configuration values provided by user in Config.Values

Create Service for mautrix-signal
(Edit Synapse ConfigMap with Synapse IP)

Edit mautrix-signal ConfigMap with mautrix-signal IP and Synapse IP

Create Deployment for mautrix-signal

Watches SYNAPSE CRD:

If synapse deleted
    Delete mautrix-signal associated to it, if any

States

Add a "waiting for Synape" state for the bridges ? to be used as sync mechanism between bridge and Synapse
What is actually the risk of configuring Synapse with a particular app service if this app service is not reachable / working / misconfigured .. ?

The design of meaningful state is discussed in #42