Implement gRPC delegation in Local Hub

[mszostok]

Description

After Investigate existing solutions to speed up Local Hub rewrite task, we decided to give up with Local Hub rewrite. Instead, we will implement our delegated storage in separate Go service and use gRPC in Local Hub to delegate values resolution.

The initial protobufs were defined in #644

Steps

• Distinguish if backend is built-in or not:
  • if external, execute gRPC call to save/lock/unlock/update/delete a given value. (for now context is ignored, will be done in Handle multiple backends for the TypeInstance upload from workflow #634)
  • ensure that transaction is aborted in case of any error
  • store returned TypeInstance.backend.context if not empty

    Q: what is the contract here? should delegated storage always return context, even if not modified or not?

• Refactor Local Hub structure to enable its extensibility.
• Refactor the createTypeInstance mutation to use the createTypeInstances. If not possible, remove it and adjust all its usage.
• Refactor Local Hub and extract logic for backend resolution:
  

  capact/hub-js/graphql/local/schema.graphql

  Lines 386 to 401 in f6f487e

  	// Backend
  	WITH *
  	CALL apoc.do.when(
  	$in.backend.id IS NOT NULL,
  	'
  	WITH false as abstract
  	RETURN $in.backend.id as id, abstract
  	',
  	'
  	// TODO(storage): this should be resolved by Local Hub server during the insertion, not in cypher.
  	WITH true as abstract
  	MATCH (backend:TypeInstance)-[:OF_TYPE]->(typeRef {path: "cap.core.type.hub.storage.neo4j"})
  	RETURN backend.id as id, abstract
  	',
  	{in: $in}
  	) YIELD value as backend

• Refactor Local Hub and store value only for built-in storage backends
• Execute gRPC call to:
  • create
  • lock/unlock
  • update
  • delete

Additional

• Update Local Hub GraphQL mutations and queries with backend.context property. It should be of type Any!. For example:

  • Mutation

    mutation CreateTypeInstances {
      createTypeInstances(
        in: {
         typeInstances: [
          {
            alias: "helm-release"
            typeRef: { path: "cap.type.helm.chart.release", revision: "0.1.0" }
            value: {
              key: "test" # same as it was
            }
            backend: {
              id: "123" # it's like that already.
              context: { # new property of type `Any!`
                name: "release-name",
                namespace: "release-namespace",
              }
            }
          }
        ]
      }
      ) {
        id
        alias
      }
    }

  • Query

    query GetTypeInstance($typeInstanceID: ID!) {
      typeInstance(id: $typeInstanceID) {
        id
        typeRef {
          path
          revision
        }
        backend { # current backend entry
          id
          abstract
        }    
        lockedBy
        latestResourceVersion {
            resourceVersion
            createdBy
            metadata {
              attributes {
                path
                revision
              }
            }
            spec {
              value
            }
            backend {
              context { # new property of type `Any!`
                name
                namespace
              }
            }
        }
      }
    }

• Validate Storage TypeInstance value - Stretch, if not possible, extract to dedicated task. - extracted to: Add Storage TypeInstance related validation #656

• Integration tests are out of scope

Related issues

See epic #604 for reason and use cases.

[mszostok]

PRs:

• Add grpc investigation, change proto location, gen TS client, fix typo #649
  • Fix proto location website#112
• Refactor hub-js, add value mirroring for specified backend #652
• Add support for external storages on lock, unlock, update, get, and delete #654