Implement Helm storage backend logic

[mszostok]

Description

Blocked By: #645

As the Helm backend storage and allow its discovery and consumption via dedicated TypeInstance.

Subtasks

• Setup Helm storage code and manifests #670
• Implement Helm release handler for Helm storage backend #671
• Implement Helm template handler for Helm storage backend #672
• Use Helm storage in manifests #673
• Document Helm storage features #674

AC

The Helm storage backend consists of two quite independent parts:

Helm release storage

• Create Helm backend under /cmd/helm-storage-backend. Implement gRPC server. Types are already generated.
  • Support all possible releases backends (ConfigMap/Secret/etc). Maybe we will need to extend the backend.context to allow specifying it.
  • Support only GetValue method, the rest just validate data (e.g. onDelete checks whether a given Helm release).
• Helm backend Type manifests is already defined. Ensure that it is up-to-date at the end of the task.
• Keep the same shape of data: https://github.com/capactio/hub-manifests/blob/main/manifests/type/helm/chart/release.yaml

Helm template storage

A storage backend that supports helm templating:

• Implement as a part of the same binary of the Helm release storage
• Create dedicated Type for a given Storage backend TypeInstance
  - In context: go template + helm release name + namespace
• Implement templating support:
  • Template passed in context to the storage backend - it is specified and stored in Local Hub on create/update
  • The service renders the template against the manifests and values for the installed release, and returns the output
  • (Almost) the same logic that we have currently in Helm Runner

Common

• Modify Helm runner:
  - Remove output.goTemplate. As discussed with @mszostok, we'll keep it to have it as a support for further workflow steps
  - Stop supporting usage of funcs from _helpers.tpl As discussed with @mszostok, we'll keep it to have it as a support for further workflow steps
  • Update JSON Schema for Type
  • Add another output from Helm Runner - create artifact, which puts context for Helm release backend
• Add dedicated manifests (probably Action) to install Helm storage (same approach as in Configure Secret storage backend to be used with Local Hub #647)
  • Single Helm chart, one deployment, two containers and two separate services
  • The implementation produces two TypeInstances for two storage backends (helm release + helm template)
• Update pipelines for releasing Helm charts and building image
• Created dedicated documentation on our website under storage backend section:
  
  • mention two separate modules of a given storage backend (actually, they are separate storage backends)
• Create a new PostgreSQL and Mattermost Implementation revisions to showcase that the Helm storage works properly.
• Out of scope: other manifests

Related issues

See epic #604 for the reason and use cases.

[pkosiec]

Proposed subtasks:

Setup [2MD] (no dependencies)

It can be further split into two tasks, but I don't think it makes sense to do so. The most important thing is that the point no 1 needs to be done ASAP to unblock other subtasks.

1. Bootstrap Go code for the helm storage backends (no dependencies) [0.5MD]
   • Single entrypoint (directory in cmd)
   • Based on flags, a different storage backend gRPC server is run
   • Both servers (release + template) implement the same Storage Backend ProtoBuf interface (use already generated Go code)
   • Modify pipelines to build a single Docker image
2. Add dedicated Helm chart and Hub manifests to install Helm storage backends (dependency: 1, but it can be developed/tested with a usage of different docker images initially, so it can be started in parallel to 1) [1.5MD]
   • Single Helm chart, one deployment, two containers and two separate services
   • The implementation produces two TypeInstances for two storage backends (helm release backend + helm template backend) - define them
   • Update pipelines for releasing Helm charts
   • Write instruction on PR how to install such storage backend. It will be later reused in documentation
   • Base on secret storage backend, as this is pretty similar case (Configure Secret storage backend to be used with Local Hub #647)

Helm release backend implementation [1MD] (dependency: Setup/1)

1. Implement Helm release backend logic

Helm template backend implementation [2MD] (dependency: Setup/1)

1. Implement Helm template backend logic

Storage usage [3MD] (dependency: Setup)

1. Modify Helm runner (dependency: 2 - agree on Helm Release artifact shape)
   • Remove output.goTemplate.
   • Stop supporting usage of funcs from _helpers.tpl
   • Update JSON Schema for Type
   • Require Helm storage (agree on a given manifest path)
   • Update output Helm Release artifact shape (put context for Helm storage backend)
2. Create a new PostgreSQL and Mattermost Implementation revisions to showcase that the Helm storage works properly.
   • Create Helm Template artifacts on PostgreSQL install and Mattermost install level (put context for Helm storage backend)
   • Create new helm.install Impl revision
   • Create new helm.upgrade Impl revision
   • Out of scope: other manifests

Documentation [1MD] (dependency: Setup)

1. Created dedicated documentation on our website under storage backend section:
   
   • mention two separate modules of a given storage backend (actually, they are separate storage backends)
   • reuse instruction from PR description from Setup