From eeadfdadf6f598cc9130cf7b23d8932b9a221011 Mon Sep 17 00:00:00 2001 From: Meggie Ladlow Date: Fri, 5 Jun 2020 11:22:55 -0400 Subject: [PATCH 1/7] Adding a storage concepts page It very briefly covers some considerations for taking backups. --- website/data/docs-navigation.js | 1 + website/pages/docs/concepts/storage.mdx | 40 +++++++++++++++++++++++++ 2 files changed, 41 insertions(+) create mode 100644 website/pages/docs/concepts/storage.mdx diff --git a/website/data/docs-navigation.js b/website/data/docs-navigation.js index d54c1602819..218a684ba1b 100644 --- a/website/data/docs-navigation.js +++ b/website/data/docs-navigation.js @@ -30,6 +30,7 @@ export default [ 'seal', 'lease', 'auth', + 'storage', 'tokens', 'response-wrapping', 'policies', diff --git a/website/pages/docs/concepts/storage.mdx b/website/pages/docs/concepts/storage.mdx new file mode 100644 index 00000000000..eafdcb83902 --- /dev/null +++ b/website/pages/docs/concepts/storage.mdx @@ -0,0 +1,40 @@ +--- +layout: docs +page_title: Storage +sidebar_title: Storage +description: >- + Vault relies on external storage to save its information. +--- + +# Storage + +As described on our [Architecture](/docs/internals/architecture) page, Vault's storage backend is untrusted storage used purely to keep encrypted information. + +## Supported Storage Backends + +For enterprise customers, HashiCorp provides official support for Consul and Vault's Integrated Storage as storage backends. + +Many other options for storage are available with community support - see our [Storage Configuration](/docs/configuration/storage) section for more information. + +## Backups & Restores + +Due to the highly flexible nature of Vault's potential storage configurations, providing exacting guidance on backing up and restoring Vault is challenging. + +When backing up Vault, there are two pieces to consider: + +1. Vault's encrypted data in the storage backend +2. Configuration files and management scripts for running the Vault server + +### Vault's Persisted Data + +In an ideal world, backups and restores would be performed while Vault is offline. If offline backups are impossible, we have recommended using a storage backend that supports atomic snapshots (such as [Consul](https://www.consul.io/docs/commands/snapshot.html) or [Integrated Storage](https://www.vaultproject.io/docs/commands/operator/raft#snapshot)). + +**If your storage backend does not support atomic snapshots, we recommend only taking offline backups.** + +We recommend taking backups **before**, but not during, write operations to the `/sys` API (excluding the `/sys/leases`, `/sys/namespaces`, `/sys/tools`, `/sys/wrapping`, `/sys/policies`, and `/sys/pprof` endpoints). Some examples of workflows that write to the `/sys` API are upgrades, server moves, and rekeys. In the future, this guidance may change for the Integrated Storage backend. + +To perform a backup or restore of Vault's encrypted data, refer to the snapshotting documentation for your storage backend. + +### Configuration + +In addition to backing up Vault's encrypted data via the storage backend, you may also wish to save the server configuration files and any scripts for managing the Vault service. The location of these files will be specific to your installation of Vault. From b30b2c71a3f67e5428eeb98cf507d66b646286aa Mon Sep 17 00:00:00 2001 From: Meggie Date: Mon, 10 Aug 2020 17:57:11 -0400 Subject: [PATCH 2/7] Apply suggestions from code review Co-authored-by: Calvin Leung Huang Co-authored-by: Jim Kalafut --- website/pages/docs/concepts/storage.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/website/pages/docs/concepts/storage.mdx b/website/pages/docs/concepts/storage.mdx index eafdcb83902..8e08e6d37f9 100644 --- a/website/pages/docs/concepts/storage.mdx +++ b/website/pages/docs/concepts/storage.mdx @@ -18,7 +18,7 @@ Many other options for storage are available with community support - see our [S ## Backups & Restores -Due to the highly flexible nature of Vault's potential storage configurations, providing exacting guidance on backing up and restoring Vault is challenging. +Due to the highly flexible nature of Vault's potential storage configurations, providing exact guidance on backing up and restoring Vault is challenging. When backing up Vault, there are two pieces to consider: @@ -27,7 +27,7 @@ When backing up Vault, there are two pieces to consider: ### Vault's Persisted Data -In an ideal world, backups and restores would be performed while Vault is offline. If offline backups are impossible, we have recommended using a storage backend that supports atomic snapshots (such as [Consul](https://www.consul.io/docs/commands/snapshot.html) or [Integrated Storage](https://www.vaultproject.io/docs/commands/operator/raft#snapshot)). +Backups and restores are ideally performed while Vault is offline. If offline backups are not feasible, we recommend using a storage backend that supports atomic snapshots (such as [Consul](https://www.consul.io/docs/commands/snapshot.html) or [Integrated Storage](https://www.vaultproject.io/docs/commands/operator/raft#snapshot)). **If your storage backend does not support atomic snapshots, we recommend only taking offline backups.** From 812cc6c8ae618f32d62786aff9203dbc1d24d25d Mon Sep 17 00:00:00 2001 From: Meggie Ladlow Date: Tue, 11 Aug 2020 15:20:10 -0400 Subject: [PATCH 3/7] Updated with some additional comments --- website/pages/docs/concepts/storage.mdx | 42 ++++++++++++++++++++++--- 1 file changed, 38 insertions(+), 4 deletions(-) diff --git a/website/pages/docs/concepts/storage.mdx b/website/pages/docs/concepts/storage.mdx index 8e08e6d37f9..e2d00a0742a 100644 --- a/website/pages/docs/concepts/storage.mdx +++ b/website/pages/docs/concepts/storage.mdx @@ -25,16 +25,50 @@ When backing up Vault, there are two pieces to consider: 1. Vault's encrypted data in the storage backend 2. Configuration files and management scripts for running the Vault server +There's also a big question - what is the error case you're trying to guard against by saving a backup? + +### The Big Question - Why Take Backups? + +It's important to consider the question of "why take a backup" while developing your ongoing +backup strategy. + +We do not recommend backups as protection against the failure of an individual machine or data +center. Vault servers can run in clusters. Vault Enterprise supports replicated clusters for +performance and disaster recovery. With open source features, a Vault cluster can extend across +multiple availability zones within a region. + +As you develop a plan for backing up and restoring Vault, include clustering in your plan to +address redundancy concerns. + ### Vault's Persisted Data -Backups and restores are ideally performed while Vault is offline. If offline backups are not feasible, we recommend using a storage backend that supports atomic snapshots (such as [Consul](https://www.consul.io/docs/commands/snapshot.html) or [Integrated Storage](https://www.vaultproject.io/docs/commands/operator/raft#snapshot)). +Backups and restores are ideally performed while Vault is offline. If offline backups are not +feasible, we recommend using a storage backend that supports atomic snapshots (such as +[Consul](https://www.consul.io/docs/commands/snapshot.html) or [Integrated +Storage](/docs/commands/operator/raft#snapshot)). **If your storage backend does not support atomic snapshots, we recommend only taking offline backups.** -We recommend taking backups **before**, but not during, write operations to the `/sys` API (excluding the `/sys/leases`, `/sys/namespaces`, `/sys/tools`, `/sys/wrapping`, `/sys/policies`, and `/sys/pprof` endpoints). Some examples of workflows that write to the `/sys` API are upgrades, server moves, and rekeys. In the future, this guidance may change for the Integrated Storage backend. +We recommend taking backups **before**, but not during, write operations to the `/sys` API +(excluding the `/sys/leases`, `/sys/namespaces`, `/sys/tools`, `/sys/wrapping`, `/sys/policies`, and +`/sys/pprof` endpoints). Some examples of workflows that write to the `/sys` API are upgrades, +server moves, and rekeys. In the future, this guidance may change for the Integrated Storage +backend. + +To perform a backup or restore of Vault's encrypted data when using a HashiCorp-supported storage +backend, see the instructions linked below. For other storage backends, follow the documentation of +that backend for taking and restoring backups. -To perform a backup or restore of Vault's encrypted data, refer to the snapshotting documentation for your storage backend. +* Integrated Storage [snapshots](/docs/commands/operator/raft#snapshot) +* Consul [snapshots](https://www.consul.io/docs/commands/snapshot.html) ### Configuration -In addition to backing up Vault's encrypted data via the storage backend, you may also wish to save the server configuration files and any scripts for managing the Vault service. The location of these files will be specific to your installation of Vault. +In addition to backing up Vault's encrypted data via the storage backend, you may also wish to +save the server configuration files, any scripts for managing the Vault service, and ensure you +can reinstall any user-installed plugins. The location of these files will be specific to your +installation of Vault. + +~> **NOTE**: Some of your configuration may be sensitive (a Vault token for Transit Autounseal +or a TLS private key in your configuration, for example). The presence of this information in +your backups will mean that they may need to be carefully protected. From 4d4253a8eca6072a2b8a15b26f2c0c3d97ae32d2 Mon Sep 17 00:00:00 2001 From: Meggie Ladlow Date: Thu, 1 Oct 2020 13:22:00 -0400 Subject: [PATCH 4/7] Attempt to further clarify sensitivity --- website/pages/docs/concepts/storage.mdx | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/website/pages/docs/concepts/storage.mdx b/website/pages/docs/concepts/storage.mdx index e2d00a0742a..52dc5cb348f 100644 --- a/website/pages/docs/concepts/storage.mdx +++ b/website/pages/docs/concepts/storage.mdx @@ -69,6 +69,7 @@ save the server configuration files, any scripts for managing the Vault service, can reinstall any user-installed plugins. The location of these files will be specific to your installation of Vault. -~> **NOTE**: Some of your configuration may be sensitive (a Vault token for Transit Autounseal +~> **NOTE**: Although a backup or snapshot of Vault's data from the storage backend is encrypted, +some of your configuration may be sensitive (a Vault token for Transit Autounseal or a TLS private key in your configuration, for example). The presence of this information in your backups will mean that they may need to be carefully protected. From 1d05de225ee74aa77380fb89fdd044eb2550529f Mon Sep 17 00:00:00 2001 From: Meggie Date: Thu, 22 Oct 2020 12:12:22 -0400 Subject: [PATCH 5/7] Update storage.mdx --- website/pages/docs/concepts/storage.mdx | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/website/pages/docs/concepts/storage.mdx b/website/pages/docs/concepts/storage.mdx index 52dc5cb348f..b0c6b4ab4fc 100644 --- a/website/pages/docs/concepts/storage.mdx +++ b/website/pages/docs/concepts/storage.mdx @@ -51,9 +51,8 @@ Storage](/docs/commands/operator/raft#snapshot)). We recommend taking backups **before**, but not during, write operations to the `/sys` API (excluding the `/sys/leases`, `/sys/namespaces`, `/sys/tools`, `/sys/wrapping`, `/sys/policies`, and -`/sys/pprof` endpoints). Some examples of workflows that write to the `/sys` API are upgrades, -server moves, and rekeys. In the future, this guidance may change for the Integrated Storage -backend. +`/sys/pprof` endpoints). Some examples of workflows that write to the `/sys` API are upgrades and +rekeys. In the future, this guidance may change for the Integrated Storage backend. To perform a backup or restore of Vault's encrypted data when using a HashiCorp-supported storage backend, see the instructions linked below. For other storage backends, follow the documentation of From ef9ad7521116a8ee2beafd247d5d7e7cddc49c44 Mon Sep 17 00:00:00 2001 From: Meggie Ladlow Date: Thu, 4 Feb 2021 14:02:49 -0500 Subject: [PATCH 6/7] More on "Why backup?" Added HashiConf talk link. --- website/content/docs/concepts/storage.mdx | 110 ++++++++++++++++++++++ website/pages/docs/concepts/storage.mdx | 74 --------------- 2 files changed, 110 insertions(+), 74 deletions(-) create mode 100644 website/content/docs/concepts/storage.mdx delete mode 100644 website/pages/docs/concepts/storage.mdx diff --git a/website/content/docs/concepts/storage.mdx b/website/content/docs/concepts/storage.mdx new file mode 100644 index 00000000000..d4280705fe0 --- /dev/null +++ b/website/content/docs/concepts/storage.mdx @@ -0,0 +1,110 @@ +--- +layout: docs +page_title: Storage +sidebar_title: Storage +description: >- + Vault relies on external storage to save its information. +--- + +# Storage + +As described on our [Architecture](/docs/internals/architecture) page, Vault's +storage backend is untrusted storage used purely to keep encrypted information. + +## Supported Storage Backends + +For enterprise customers, HashiCorp provides official support for Consul and +Vault's Integrated Storage as storage backends. + +Many other options for storage are available with community support - see our +[Storage Configuration](/docs/configuration/storage) section for more +information. + +## Backups + +Due to the highly flexible nature of Vault's potential storage configurations, +providing exact guidance on backing up Vault is challenging. + +When backing up Vault, there are two pieces to consider: + +1. Vault's encrypted data in the storage backend +2. Configuration files and management scripts for running the Vault server + +There's also a big question - what is the error case you're trying to guard +against by saving a backup? + +### The Big Question - Why Take Backups? + +It's important to consider the question of "why take a backup" while developing +your ongoing backup and disaster recovery strategy. + +Taking a backup is recommended prior to upgrades, as downgrading Vault storage +is not always possible. Generally, a backup is recommended any time a major +change is planned for a cluster. + +More specifically, we recommend taking backups **before**, but not during, write +operations to the `/sys` API (excluding the `/sys/leases`, `/sys/namespaces`, +`/sys/tools`, `/sys/wrapping`, `/sys/policies`, and `/sys/pprof` endpoints). +Some examples of workflows that write to the `/sys` API are upgrades and rekeys. +In the future, this guidance may change for the Integrated Storage backend. + +Backups _can_ also help with accidential data deletions or modifications. In +this case, the story can get a little tricky. If you simply recover a backup +from 5AM with the correct data, but the current time is 10AM, you will lose data +written between 5 and 10AM. Lucy Davinhart gave a HashiConf talk that serves as +an interesting [case +study](https://www.hashicorp.com/resources/oh-no-i-deleted-my-vault-secret). + +We do not recommend backups as protection against the failure of an individual +machine. Vault servers can run in clusters, so to protect against server +failure, we recommend running Vault in [HA +mode](/docs/internals/high-availability). With open source features, a +Vault cluster can extend across multiple availability zones within a region. + +Vault Enterprise supports replicated clusters and disaster recovery for data +center failure. When using OSS Vault in [HA +Mode](/docs/internals/high-availability), a backup can help guard against the +failure of a data center. + +Ultimately, backups are not a replacement for running in HA, or for using +replication with Vault Enterprise. As you develop a plan for recovering from or +guarding against failure, you should consider both backups and HA as critical +components of that plan. + +### Backing up Vault's Persisted Data + +Backups and restores are ideally performed while Vault is offline. If offline +backups are not feasible, we recommend using a storage backend that supports +atomic snapshots (such as +[Consul](https://www.consul.io/docs/commands/snapshot.html) or [Integrated +Storage](/docs/commands/operator/raft#snapshot)). + +~> If your storage backend does not support atomic snapshots, we recommend only +taking offline backups. + +To perform a backup or restore of Vault's encrypted data when using a +HashiCorp-supported storage backend, see the instructions linked below. For +other storage backends, follow the documentation of that backend for taking and +restoring backups. + +* Integrated Storage [snapshots](/docs/commands/operator/raft#snapshot) +* Consul [snapshots](https://www.consul.io/docs/commands/snapshot.html) + +#### Backing up Multiple Clusters + +If you are using Vault Enterprise [Performance +Replication](/docs/enterprise/replication#performance-replication-and-disaster-recovery-dr-replication), +you should plan to take backups of the active node on each of your clusters. + +### Configuration + +In addition to backing up Vault's encrypted data via the storage backend, you +may also wish to save the server configuration files, any scripts for managing +the Vault service, and ensure you can reinstall any user-installed plugins. The +location of these files will be specific to your installation of Vault. + +~> **NOTE**: Although a backup or snapshot of Vault's data from the storage +backend is encrypted, some of your configuration may be sensitive (a Vault token +for Transit Autounseal or a TLS private key in your configuration, for example). +The presence of this information in your backups will mean that they may need +to be carefully protected. diff --git a/website/pages/docs/concepts/storage.mdx b/website/pages/docs/concepts/storage.mdx deleted file mode 100644 index b0c6b4ab4fc..00000000000 --- a/website/pages/docs/concepts/storage.mdx +++ /dev/null @@ -1,74 +0,0 @@ ---- -layout: docs -page_title: Storage -sidebar_title: Storage -description: >- - Vault relies on external storage to save its information. ---- - -# Storage - -As described on our [Architecture](/docs/internals/architecture) page, Vault's storage backend is untrusted storage used purely to keep encrypted information. - -## Supported Storage Backends - -For enterprise customers, HashiCorp provides official support for Consul and Vault's Integrated Storage as storage backends. - -Many other options for storage are available with community support - see our [Storage Configuration](/docs/configuration/storage) section for more information. - -## Backups & Restores - -Due to the highly flexible nature of Vault's potential storage configurations, providing exact guidance on backing up and restoring Vault is challenging. - -When backing up Vault, there are two pieces to consider: - -1. Vault's encrypted data in the storage backend -2. Configuration files and management scripts for running the Vault server - -There's also a big question - what is the error case you're trying to guard against by saving a backup? - -### The Big Question - Why Take Backups? - -It's important to consider the question of "why take a backup" while developing your ongoing -backup strategy. - -We do not recommend backups as protection against the failure of an individual machine or data -center. Vault servers can run in clusters. Vault Enterprise supports replicated clusters for -performance and disaster recovery. With open source features, a Vault cluster can extend across -multiple availability zones within a region. - -As you develop a plan for backing up and restoring Vault, include clustering in your plan to -address redundancy concerns. - -### Vault's Persisted Data - -Backups and restores are ideally performed while Vault is offline. If offline backups are not -feasible, we recommend using a storage backend that supports atomic snapshots (such as -[Consul](https://www.consul.io/docs/commands/snapshot.html) or [Integrated -Storage](/docs/commands/operator/raft#snapshot)). - -**If your storage backend does not support atomic snapshots, we recommend only taking offline backups.** - -We recommend taking backups **before**, but not during, write operations to the `/sys` API -(excluding the `/sys/leases`, `/sys/namespaces`, `/sys/tools`, `/sys/wrapping`, `/sys/policies`, and -`/sys/pprof` endpoints). Some examples of workflows that write to the `/sys` API are upgrades and -rekeys. In the future, this guidance may change for the Integrated Storage backend. - -To perform a backup or restore of Vault's encrypted data when using a HashiCorp-supported storage -backend, see the instructions linked below. For other storage backends, follow the documentation of -that backend for taking and restoring backups. - -* Integrated Storage [snapshots](/docs/commands/operator/raft#snapshot) -* Consul [snapshots](https://www.consul.io/docs/commands/snapshot.html) - -### Configuration - -In addition to backing up Vault's encrypted data via the storage backend, you may also wish to -save the server configuration files, any scripts for managing the Vault service, and ensure you -can reinstall any user-installed plugins. The location of these files will be specific to your -installation of Vault. - -~> **NOTE**: Although a backup or snapshot of Vault's data from the storage backend is encrypted, -some of your configuration may be sensitive (a Vault token for Transit Autounseal -or a TLS private key in your configuration, for example). The presence of this information in -your backups will mean that they may need to be carefully protected. From 723ef1ba5200f4650a9d5be0b1460204e25481cd Mon Sep 17 00:00:00 2001 From: Meggie Date: Tue, 16 Feb 2021 17:30:48 -0500 Subject: [PATCH 7/7] Update website/content/docs/concepts/storage.mdx Co-authored-by: Vishal Nayak --- website/content/docs/concepts/storage.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/content/docs/concepts/storage.mdx b/website/content/docs/concepts/storage.mdx index d4280705fe0..2cbf2eeedff 100644 --- a/website/content/docs/concepts/storage.mdx +++ b/website/content/docs/concepts/storage.mdx @@ -3,7 +3,7 @@ layout: docs page_title: Storage sidebar_title: Storage description: >- - Vault relies on external storage to save its information. + Vault relies on external storage to save its durable information. --- # Storage