From 98ee63de05af6757353dc76fa3ee77988fde128b Mon Sep 17 00:00:00 2001 From: Pawel Kosiec Date: Mon, 7 Mar 2022 17:58:43 +0100 Subject: [PATCH 1/3] Describe AWS Secrets Storage backend --- docs/feature/storage-backends/_category_.json | 4 + .../storage-backends/aws-secrets-manager.md | 165 ++++++++++++++++++ docs/feature/storage-backends/introduction.md | 7 + 3 files changed, 176 insertions(+) create mode 100644 docs/feature/storage-backends/_category_.json create mode 100644 docs/feature/storage-backends/aws-secrets-manager.md create mode 100644 docs/feature/storage-backends/introduction.md diff --git a/docs/feature/storage-backends/_category_.json b/docs/feature/storage-backends/_category_.json new file mode 100644 index 0000000..133efaf --- /dev/null +++ b/docs/feature/storage-backends/_category_.json @@ -0,0 +1,4 @@ +{ + "label": "Storage Backends", + "position": 2 +} diff --git a/docs/feature/storage-backends/aws-secrets-manager.md b/docs/feature/storage-backends/aws-secrets-manager.md new file mode 100644 index 0000000..b83c215 --- /dev/null +++ b/docs/feature/storage-backends/aws-secrets-manager.md @@ -0,0 +1,165 @@ +# AWS Secrets Manager + +You can use [AWS Secrets Manager](https://aws.amazon.com/secrets-manager/) service to store and manage your TypeInstance values externally. The functionality in a form of a dedicated storage backend is installed using Capact Action. This guide describes how to install and configure the AWS Secrets Manager storage backend for Capact Local Hub. + +To learn more about Capact Local Hub storage backends, see the [Introduction](./introduction.md) document. + +## Features + +AWS Secrets Manager storage backend is feature-complete. It supports full lifecycle of TypeInstance: create, get, update, delete, lock, unlock. + +It can be used as a default storage. For more information, see the [Use the storage backend](#use-the-storage-backend) paragraph. + +## Prerequisites + +* [Capact CLI](../../cli/getting-started.mdx) installed. +* Cluster with Capact installation. See the [installation tutorial](../../installation/local.mdx). +* Access to AWS + +## Obtain AWS credentials + +Create AWS security credentials with `SecretsManagerReadWrite` policy. + +1. In AWS dashboard, navigate to IAM (access management). + +1. In **Users** tab, add a new user. +1. Select **Access key** AWS credential type. +1. On **Set permissions** step, select **Attach existing policies directly***. +1. Find **SecretsManagerReadWrite** policy. +1. Go to next step until you see the **Create user** step. Click on it. +1. Copy the credentials and follow the [Installation](#installation) guide. + +## Installation + +1. Export the environment variables: + + ```bash + export AWS_ACCESS_KEY_ID="{AWS access key ID}" + export AWS_SECRET_ACCESS_KEY="{AWS secret access key}" + ``` + +1. Follow the [AWS Credentials TypeInstance creation](../../example/typeinstances.md#aws-credentials) guide to create and obtain ID of the newly created TypeInstance. + + Export it as `TI_ID` environment variable: + + ```bash + export TI_ID="{id}" + ``` + +1. Update Capact Global policy to inject the AWS credentials when necessary: + + ```bash + cat > /tmp/aws-storage-policy.yaml << ENDOFFILE + interface: + rules: + - interface: + path: cap.interface.aws.secrets-manager.storage.install + oneOf: + - implementationConstraints: + attributes: + - path: "cap.attribute.cloud.provider.aws" + inject: + requiredTypeInstances: + - id: ${TI_ID} + description: "AWS credentials" + - interface: + path: cap.* + oneOf: + - implementationConstraints: + requires: + - path: cap.core.type.platform.kubernetes + - implementationConstraints: {} + typeInstance: + rules: [] + ENDOFFILE + capact policy apply -f /tmp/aws-storage-policy.yaml + ``` + +1. Create parameters: + + ```bash + cat > /tmp/aws-storage-input-params.yaml << ENDOFFILE + input-parameters: + region: "eu-west-1" # AWS region, where the secrets are managed + ENDOFFILE + ``` + +1. Create Action: + ```bash + capact act create cap.interface.aws.secrets-manager.storage.install --name aws-storage -n capact-system --parameters-from-file /tmp/aws-storage-input-params.yaml + ``` + +1. Wait for the Action to have the `READY_TO_RUN` status: + + ```bash + capact act get aws-storage -n capact-system + ``` + +1. Run Action: + + ```bash + capact act run aws-storage -n capact-system + ``` + +1. Watch the Action for completion: + + ```bash + capact act watch aws-storage -n capact-system + ``` + +1. Get output TypeInstances: + + ```bash + capact act get aws-storage -n capact-system -ojson | jq '.Actions[0].output.typeInstances' + ``` + + See the details of the installed AWS storage backend: + + ```bash + AWS_SM_STORAGE_ID=$(capact act get aws-storage -n capact-system -ojson | jq '.Actions[0].output.typeInstances | map(select(.typeRef.path == "cap.type.aws.secrets-manager.storage"))[0].id' -r) + capact ti get $AWS_SM_STORAGE_ID -oyaml + ``` + +## Use the storage backend + +Once the storage backend is installed, you can configure it for selected set of TypeInstances. + +For example, you can configure it as a default backend for all TypeInstances with the following Global Policy: + +```bash +cat > /tmp/aws-storage-policy.yaml << ENDOFFILE +interface: + rules: + - interface: + path: cap.* + oneOf: + - implementationConstraints: + requires: + - path: cap.core.type.platform.kubernetes + - implementationConstraints: {} +typeInstance: + rules: + - typeRef: + path: "cap.*" + backend: + id: "${AWS_SM_STORAGE_ID}" + description: "AWS Secrets Manager" +ENDOFFILE +capact policy apply -f /tmp/aws-storage-policy.yaml +``` + +To read more about TypeInstance backend configuration, see the [Definition of rules for TypeInstance](../policies/overview.md#definition-of-rules-for-typeinstance) section. + +## Next steps + +Now, you can run any Capact manifest and utilize the newly installed AWS Secrets Manager storage backend. For example, [install Mattermost](../../example/mattermost-installation.md). Once it's installed, verify the output TypeInstance values are created in AWS Secrets Manager. You can do it in two ways: + +- verify the `backend.id` for output TypeInstances of a given Action: + + ```bash + capact act get {action Name} -ojson | jq '.Actions[0].output.typeInstances'` + ``` + +- see the AWS Secrets Manager UI to verify the TypeInstance value is stored externally. + +To clean up the secrets stored externally, just delete a given TypeInstance. \ No newline at end of file diff --git a/docs/feature/storage-backends/introduction.md b/docs/feature/storage-backends/introduction.md new file mode 100644 index 0000000..76a2512 --- /dev/null +++ b/docs/feature/storage-backends/introduction.md @@ -0,0 +1,7 @@ +--- +sidebar_position: 1 +--- + +# Introduction + + From 741eda77d5da9ac0c78c0473d25ef9b2efddf587 Mon Sep 17 00:00:00 2001 From: Pawel Kosiec Date: Fri, 11 Mar 2022 13:27:45 +0100 Subject: [PATCH 2/3] Add introduction --- .../storage-backends/aws-secrets-manager.md | 2 +- docs/feature/storage-backends/introduction.md | 24 +++++++++++++++++++ 2 files changed, 25 insertions(+), 1 deletion(-) diff --git a/docs/feature/storage-backends/aws-secrets-manager.md b/docs/feature/storage-backends/aws-secrets-manager.md index b83c215..f75265d 100644 --- a/docs/feature/storage-backends/aws-secrets-manager.md +++ b/docs/feature/storage-backends/aws-secrets-manager.md @@ -8,7 +8,7 @@ To learn more about Capact Local Hub storage backends, see the [Introduction](./ AWS Secrets Manager storage backend is feature-complete. It supports full lifecycle of TypeInstance: create, get, update, delete, lock, unlock. -It can be used as a default storage. For more information, see the [Use the storage backend](#use-the-storage-backend) paragraph. +It can be used as a default storage for all TypeInstances. ## Prerequisites diff --git a/docs/feature/storage-backends/introduction.md b/docs/feature/storage-backends/introduction.md index 76a2512..b330f06 100644 --- a/docs/feature/storage-backends/introduction.md +++ b/docs/feature/storage-backends/introduction.md @@ -4,4 +4,28 @@ sidebar_position: 1 # Introduction +By default, Capact stores the state in a form of TypeInstances, that contain static data. The data is stored unencrypted in the same Capact Local Hub database. +However, there are many cases, where static, unencrypted data is not enough. For example, some sensitive data should be stored in a secure way. Or, Helm chart details may quickly become outdated in case of any external change (e.g. upgrade with `helm` CLI). + +That's why Capact introduced storage backends functionality. Storage backend is a service which manages external TypeInstance's `value` for every resource version, as well as the `lockedBy` piece of information. + +It supports full TypeInstance lifecycle, that is: create, get, update, delete, lock, unlock operations. + + +## Available storage backends + +Currently, we have the following storage backends available: + +- [AWS Secrets Manager](./aws-secrets-manager.md) + +Each storage backend is installed separately using Capact Actions. See corresponding documents for each storage to learn how to install and configure them. + +## Development + +To develop your own storage backend, create a gRPC service which implements the [storage backend Protocol Buffers schema](https://github.com/capactio/capact/blob/main/hub-js/proto/storage_backend.proto). + +You can use the generated server code for the following languages: +- [Go](https://github.com/capactio/capact/tree/main/pkg/hub/api/grpc/storage_backend), +- [TypeScript (Node.js)](https://github.com/capactio/capact/tree/main/hub-js/src/generated/grpc), +- or, generate your own. From 8c0352b20807bd984d522773b2010d66e0ce846f Mon Sep 17 00:00:00 2001 From: Pawel Kosiec Date: Sun, 13 Mar 2022 16:52:43 +0100 Subject: [PATCH 3/3] Improve documents after review --- .../storage-backends/aws-secrets-manager.md | 16 ++++++++-------- docs/feature/storage-backends/introduction.md | 2 +- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/feature/storage-backends/aws-secrets-manager.md b/docs/feature/storage-backends/aws-secrets-manager.md index f75265d..f49f13c 100644 --- a/docs/feature/storage-backends/aws-secrets-manager.md +++ b/docs/feature/storage-backends/aws-secrets-manager.md @@ -26,7 +26,7 @@ Create AWS security credentials with `SecretsManagerReadWrite` policy. 1. Select **Access key** AWS credential type. 1. On **Set permissions** step, select **Attach existing policies directly***. 1. Find **SecretsManagerReadWrite** policy. -1. Go to next step until you see the **Create user** step. Click on it. +1. Go to the next step until you see the **Create user** step. Click on it. 1. Copy the credentials and follow the [Installation](#installation) guide. ## Installation @@ -46,7 +46,7 @@ Create AWS security credentials with `SecretsManagerReadWrite` policy. export TI_ID="{id}" ``` -1. Update Capact Global policy to inject the AWS credentials when necessary: +1. Update the [Global policy](../policies/global-policy.md) to inject the AWS credentials for the storage backend installation: ```bash cat > /tmp/aws-storage-policy.yaml << ENDOFFILE @@ -152,14 +152,14 @@ To read more about TypeInstance backend configuration, see the [Definition of ru ## Next steps -Now, you can run any Capact manifest and utilize the newly installed AWS Secrets Manager storage backend. For example, [install Mattermost](../../example/mattermost-installation.md). Once it's installed, verify the output TypeInstance values are created in AWS Secrets Manager. You can do it in two ways: +Now, you can run any Capact manifest and utilize the newly installed AWS Secrets Manager storage backend. For example, [install Mattermost](../../example/mattermost-installation.md). Once it is installed, verify the output TypeInstance values are created in AWS Secrets Manager. You can do it in the following ways: -- verify the `backend.id` for output TypeInstances of a given Action: + - verify the `backend.id` for output TypeInstances of a given Action: - ```bash - capact act get {action Name} -ojson | jq '.Actions[0].output.typeInstances'` - ``` + ```bash + capact act get {action Name} -ojson | jq '.Actions[0].output.typeInstances'` + ``` -- see the AWS Secrets Manager UI to verify the TypeInstance value is stored externally. + - see the AWS Secrets Manager UI to verify the TypeInstance value is stored externally. To clean up the secrets stored externally, just delete a given TypeInstance. \ No newline at end of file diff --git a/docs/feature/storage-backends/introduction.md b/docs/feature/storage-backends/introduction.md index b330f06..58b7840 100644 --- a/docs/feature/storage-backends/introduction.md +++ b/docs/feature/storage-backends/introduction.md @@ -28,4 +28,4 @@ To develop your own storage backend, create a gRPC service which implements the You can use the generated server code for the following languages: - [Go](https://github.com/capactio/capact/tree/main/pkg/hub/api/grpc/storage_backend), - [TypeScript (Node.js)](https://github.com/capactio/capact/tree/main/hub-js/src/generated/grpc), -- or, generate your own. +- or generate your own based on the Protocol Buffers schema.