From 0da59962d029baaa5211db503b6f44c2ae451976 Mon Sep 17 00:00:00 2001 From: Yi Wu Date: Mon, 27 Apr 2020 12:29:19 +0800 Subject: [PATCH 01/30] encryption-at-rest docs Signed-off-by: Yi Wu --- .gitignore | 1 + reference/security/encryption-at-rest.md | 116 +++++++++++++++++++++++ 2 files changed, 117 insertions(+) create mode 100644 reference/security/encryption-at-rest.md diff --git a/.gitignore b/.gitignore index fd9ffe0c2b502..35a883df07476 100644 --- a/.gitignore +++ b/.gitignore @@ -8,5 +8,6 @@ out gen .DS_Store +*.swp /node_modules/ diff --git a/reference/security/encryption-at-rest.md b/reference/security/encryption-at-rest.md new file mode 100644 index 0000000000000..de2884d66c67e --- /dev/null +++ b/reference/security/encryption-at-rest.md @@ -0,0 +1,116 @@ +--- +title: Encryption-At-Rest +summary: Learn how to enable encryption-at-rest to protect sensitive data. +category: reference +--- + +# Encryption-At-Rest for TiKV New in v4.0.0 + +Encryption-at-rest means that data is encrypted when it is stored. As opposed to encryption in flight (TLS) or encryption in use (rarely used). Different things could be doing encryption-at-rest (SSD drive, file system, cloud vendor, etc), but if the database does encryption by itself this helps ensure that attackers must authenticate with the database to gain access to data. For example, even when an attacker gains access to the physical machine, he cannot access data by copying files on disk. + +TiKV supports encryption-at-rest starting from v4.0.0. The feature allows TiKV to transparently encrypt data files using [AES](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard) in [CTR](https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation) mode. To enable encryption-at-rest, an encryption key must be provided by user and this key is called master key. The suggested way to provide the master key is via AWS KMS, but specifying a key in plaintext stored in a file is also supported. TiKV automatically rotate data keys that it used to encrypt actual data files. Manually rotating the master key can be done occassionally. Note that encryption-at-rest only encrypts data at rest (i.e. on disk) and not while data is transferred over network. It is advices to use TLS together with encryption-at-rest. + +Also from v4.0.0, BR supports S3 server-side encryption (SSE) when backup to S3. A customer owned AWS KMS key can also be used together with S3 server-side encrytion. + +## TiKV Encryption-At-Rest + +### Overview + +TiKV currently support encrypting data using AES128, AES192 or AES256, in CTR mode. There are two types of keys used in TiKV when encryption is enabled: + +1. Master key. The master key is provided by user and is used to encrypt the data keys TiKV generates. Management of master key is external to TiKV. +2. Data key. The data key is generated by TiKV and is the key actually being used to encrypt data. The data key is automatically rotated by TiKV. + +The same master key can be shared by multiple instances of TiKV. The recommended way to provide a master key in production is via AWS KMS. User would need to create a CMK through AWS KMS, and then provide the CMK key id to TiKV in config file. The TiKV process would need access to the KMS CMK while it is running, which can be done by using [IAM](https://aws.amazon.com/iam/). If TiKV fail to get access to the KMS CMK, it will failed to start or restart. If TiKV loss access to the KMS CMK while it is running, data key rotation will be temporarily disabled. Please refer to AWS documentation for KMS and IAM usage. + +Alternatively, if using custom key is desired, supplying master key via file is also supported. The file need to contain a 256 bits (or 32 bytes) key encoded as hex string. The file should end with a newline (i.e. "\n") and contain nothing else. Persistenting the key on disk, however, leaks the key, so the key file is only suitable to be stored on tempfs. + +Data keys are generated by TiKV and pass to underlying storage engine (i.e. RocksDB). All files written by RocksDB, including SST files, WAL files, and the MANIFEST file, is encrypted by the current data key. Other temporary files used by TiKV that may include user data are also encrypted using the same data key. By default, data keys are automatically rotated by TiKV every week, for which the period is configurable. On key rotation, TiKV does not rewrite all existing files to replace the key, but RocksDB compaction are expected to rewrite old data into new data files, with the most recent data key, if the cluster gets constant write workload. TiKV keeps track of the key and encryption method used to encrypt each of of files and use the information to decrypt the content on reads. + +Regardless of data encryption method, data keys are encrypted using AES256 in GCM mode for additional authentication. This is why when passing master key from file the key has to be of 256 bits. + +### Configuring Encryption + +To enable encryption, you can add the encryption section in TiKV's config file: + +``` +[encryption] +data-encryption-method = aes128-ctr +data-key-rotation-period = 7d +``` + +Possible values for `data-encryption-method` are "aes128-ctr", "aes-192-ctr", "aes256-ctr" and "plaintext". The default value is "plaintext", which means encryption is not turned on. `data-key-rotation-period` defines how often TiKV rotate data key. Encryption can be turned on for a fresh TiKV cluster, or an existing TiKV cluster, though only data written after encryption being enabled are guaranteed to get encrypted. To disable encryption, remove `data-encryption-method` in config file, or reset it to "plaintext", and restart TiKV. To change encryption method, update `data-encryption-method` in config file and restart TiKV. + +Master key has to be specified if encryption is enabled (i.e. `data-encryption-method` is not "plaintext"). To specify a AWS KMS CMK as master key, add the `encryption.master-key` section after the `encryption` section: + +``` +[encryption.master-key] +type = "kms" +key-id = "0987dcba-09fe-87dc-65ba-ab0987654321" +region = "us-west-2" +endpoint = "https://kms.us-west-2.amazonaws.com" +``` + +The `key-id` specifies the key id for the KMS CMK. The `region` is the AWS region name for the KMS CMK. The `endpoint` is optional and don't need to be specified normally, unless you are using a AWS KMS compatible service from a non-AWS vendor. + +To specify a master key that's stored in a file, the master key config would look like the following: + +``` +[encryption.master-key] +type = "file" +path = "/path/to/key/file" +``` + +Here `path` is the path to the key file. The file must contain a 256 bits (or 16 bytes) key encoded as hex string, end with a newline ("\n") and contain nothing else. Example of the file content: + +``` +3b5896b5be691006e0f71c3040a29495ddcad20b14aff61806940ebd780d3c62 +``` + +### Rotating Master Key + +To rotate master key, you have to specify both of the new master key and old master key in the config, and restart TiKV. Use `encryption.master-key` to specify the new master key, and use `encryption.previous-master-key` to specify the old master key. The config format for `encryption.previous-master-key` is the same as `encryption.master-key`. On restart TiKV would need access to both of the old and new master key, but once TiKV is up and running, TiKV will only need access to the new key. It is okay to leave the `encryption.previous-master-key` config in the config file from that on. Even on restart, TiKV will only try to use the old key if it fail to decrypt existing data using the new master key. + +Currently we don't support online master key rotation. Rotating master key would require restarting TiKV. + +Here is an example config for rotating the KMS CMK: + +``` +[encryption.master-key] +type = "kms" +key-id = "50a0c603-1c6f-11e6-bb9e-3fadde80ce75" +region = "us-west-2" + +[encryption.previous-master-key] +type = "kms" +key-id = "0987dcba-09fe-87dc-65ba-ab0987654321" +region = "us-west-2" +``` + +### Caveat + +The current version of TiKV encryption has some drawbacks that need to be noted when use. We are working actively to address those issues and improvments are in the pipe for future versions. + +* When deploying a TiDB cluster, the majority of user data are stored in TiKV nodes, and those data are encrypted-at-rest when encryption is enabled. However there are small amount of user data stored in PD nodes as metadata. As of v4.0.0, PD doesn't support encryption-at-rest. As a workaround, it is recommended to use storage level encryption (e.g. file system encryption) to protect sensitive data stored in PD. +* TiKV currently does not exclude encryption keys and user data from core dump. It is highly adviced to disable core dump for TiKV process when using encryption-at-rest. This is not currently handled by TiKV itself. +* TiKV keeps track of encryption key and method used for each data files, using absolute path of the files. As a result, once encryption is turned on for a TiKV node, user should not change data file paths config such as `storage.data-dir`, `raftstore.raftdb-path`, `rocksdb.wal-dir` and `raftdb.wal-dir`. + +## BR S3 server-side encryption + +To enable S3 server-side encryption when backup to S3 using BR, pass `--s3.sse` argument and set value to "aws:kms". S3 will use its own KMS key for encryption. Example: + +``` +./br backup full --pd --storage "s3:///" --s3.sse aws:kms +``` + +To use a custom AWS KMS CMK that you created and owned, pass `--s3.sse-kms-key-id` in addition. In this case, both of the BR process and all the TiKV nodes in the cluster would need access to the KMS CMK, and the KMS CMK need to be in the same AWS region as the S3 bucket used for store the backup. It is adviced to grant access to the KMS CMK to BR process and TiKV nodes via AWS IAM. Please refer to AWS documentation for usage of IAM. Example: + +``` +./br backup full --pd --storage "s3:///" --s3.region --s3.sse aws:kms --s3.sse-kms-key-id 0987dcba-09fe-87dc-65ba-ab0987654321 +``` + +On restore the backup, both `--s3.sse` and `--s3.sse-kms-key-id` should NOT be used, as S3 will figure out encryption settings by itself. The BR process and TiKV nodes in the cluster to restore the backup to would also need access to the KMS CMK, or the restore will fail. Example: + +``` +./br restore full --pd --storage "s3:/// --s3.region " +``` From 0d20b9fa6f85571327f7f53edee0d1d7e7f97284 Mon Sep 17 00:00:00 2001 From: Yi Wu Date: Wed, 13 May 2020 03:05:35 +0800 Subject: [PATCH 02/30] update encryption config name Signed-off-by: Yi Wu --- reference/security/encryption-at-rest.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/reference/security/encryption-at-rest.md b/reference/security/encryption-at-rest.md index de2884d66c67e..ac069eba02372 100644 --- a/reference/security/encryption-at-rest.md +++ b/reference/security/encryption-at-rest.md @@ -34,7 +34,7 @@ Regardless of data encryption method, data keys are encrypted using AES256 in GC To enable encryption, you can add the encryption section in TiKV's config file: ``` -[encryption] +[security.encryption] data-encryption-method = aes128-ctr data-key-rotation-period = 7d ``` @@ -44,7 +44,7 @@ Possible values for `data-encryption-method` are "aes128-ctr", "aes-192-ctr", "a Master key has to be specified if encryption is enabled (i.e. `data-encryption-method` is not "plaintext"). To specify a AWS KMS CMK as master key, add the `encryption.master-key` section after the `encryption` section: ``` -[encryption.master-key] +[security.encryption.master-key] type = "kms" key-id = "0987dcba-09fe-87dc-65ba-ab0987654321" region = "us-west-2" @@ -56,7 +56,7 @@ The `key-id` specifies the key id for the KMS CMK. The `region` is the AWS regio To specify a master key that's stored in a file, the master key config would look like the following: ``` -[encryption.master-key] +[security.encryption.master-key] type = "file" path = "/path/to/key/file" ``` @@ -76,12 +76,12 @@ Currently we don't support online master key rotation. Rotating master key would Here is an example config for rotating the KMS CMK: ``` -[encryption.master-key] +[security.encryption.master-key] type = "kms" key-id = "50a0c603-1c6f-11e6-bb9e-3fadde80ce75" region = "us-west-2" -[encryption.previous-master-key] +[security.encryption.previous-master-key] type = "kms" key-id = "0987dcba-09fe-87dc-65ba-ab0987654321" region = "us-west-2" From 80f8ba99d4d7874288109c9eb785bf09c1e5e6b4 Mon Sep 17 00:00:00 2001 From: Yi Wu Date: Wed, 13 May 2020 03:33:42 +0800 Subject: [PATCH 03/30] add monitoring section Signed-off-by: Yi Wu --- reference/security/encryption-at-rest.md | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/reference/security/encryption-at-rest.md b/reference/security/encryption-at-rest.md index ac069eba02372..c9358c5a23e56 100644 --- a/reference/security/encryption-at-rest.md +++ b/reference/security/encryption-at-rest.md @@ -87,7 +87,17 @@ key-id = "0987dcba-09fe-87dc-65ba-ab0987654321" region = "us-west-2" ``` -### Caveat +### Monitoring + +To monitor encryption-at-rest, if you deploy TiKV with Grafana, you can look at the "Encryption" pannel in "TiKV-Details" dashboard. There are a few metrics to look for: + +* Encryption initialized: 1 if encryption is initialized during TiKV startup, 0 otherwise. In case of master key rotation, after encryption is initialized, TiKV do not need access to the previous master key. +* Encryption data keys: number of existings data keys. The number is bumped by 1 after each time data key rotation happened. Use this metrics to monitor if data key rotation works as expected. +* Encrypted files: number of encrypted data files currently exists. Compare this number to existings data files in the data directory to estimate portion of data being encrypted, when turning on encryption for a previously unencrypted cluster. +* Encryption meta file size: size of the encryption meta data files. +* Read/Write encryption meta duration: the extra overhead for operate on encryption metadata. + +### Caveats The current version of TiKV encryption has some drawbacks that need to be noted when use. We are working actively to address those issues and improvments are in the pipe for future versions. From f060bfeab6e6f9e393019ca682b2e1a6d4ffe247 Mon Sep 17 00:00:00 2001 From: Yi Wu Date: Wed, 13 May 2020 09:09:08 +0800 Subject: [PATCH 04/30] address comments Signed-off-by: Yi Wu --- reference/security/encryption-at-rest.md | 49 ++++++++++++------------ 1 file changed, 25 insertions(+), 24 deletions(-) diff --git a/reference/security/encryption-at-rest.md b/reference/security/encryption-at-rest.md index c9358c5a23e56..76f39c97606a8 100644 --- a/reference/security/encryption-at-rest.md +++ b/reference/security/encryption-at-rest.md @@ -6,24 +6,33 @@ category: reference # Encryption-At-Rest for TiKV New in v4.0.0 -Encryption-at-rest means that data is encrypted when it is stored. As opposed to encryption in flight (TLS) or encryption in use (rarely used). Different things could be doing encryption-at-rest (SSD drive, file system, cloud vendor, etc), but if the database does encryption by itself this helps ensure that attackers must authenticate with the database to gain access to data. For example, even when an attacker gains access to the physical machine, he cannot access data by copying files on disk. +Encryption-at-rest means that data is encrypted when it is stored. For databases, this feature is also referred to as TDE (transparent data encryption). This is opposed to encryption in flight (TLS) or encryption in use (rarely used). Different things could be doing encryption-at-rest (SSD drive, file system, cloud vendor, etc), but by having TiKV do the encryption before storage this helps ensure that attackers must authenticate with the database to gain access to data. For example, when an attacker gains access to the physical machine, data cannot be accessed data by copying files on disk. -TiKV supports encryption-at-rest starting from v4.0.0. The feature allows TiKV to transparently encrypt data files using [AES](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard) in [CTR](https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation) mode. To enable encryption-at-rest, an encryption key must be provided by user and this key is called master key. The suggested way to provide the master key is via AWS KMS, but specifying a key in plaintext stored in a file is also supported. TiKV automatically rotate data keys that it used to encrypt actual data files. Manually rotating the master key can be done occassionally. Note that encryption-at-rest only encrypts data at rest (i.e. on disk) and not while data is transferred over network. It is advices to use TLS together with encryption-at-rest. +TiKV supports encryption-at-rest starting from v4.0.0. The feature allows TiKV to transparently encrypt data files using [AES](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard) in [CTR](https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation) mode. To enable encryption-at-rest, an encryption key must be provided by user and this key is called master key. The master key can be provided via AWS KMS (recommended), or specifying a key stored as plaintext in a file. TiKV automatically rotate data keys that it used to encrypt actual data files. Manually rotating the master key can be done occassionally. Note that encryption-at-rest only encrypts data at rest (i.e. on disk) and not while data is transferred over network. It is advices to use TLS together with encryption-at-rest. -Also from v4.0.0, BR supports S3 server-side encryption (SSE) when backup to S3. A customer owned AWS KMS key can also be used together with S3 server-side encrytion. +Also from v4.0.0, BR supports S3 server-side encryption (SSE) when backing up to S3. A customer owned AWS KMS key can also be used together with S3 server-side encrytion. + +### Caveats + +The current version of TiKV encryption has some drawbacks that we are working actively to address in the future versions. + +* When deploying a TiDB cluster, the majority of a user's data is stored in TiKV nodes, and that data will be encrypted when encryption is enabled. However there is a small amount of user data stored in PD nodes as metadata (e.g secondary index keys used as TiKV region boundaries). As of v4.0.0, PD doesn't support encryption-at-rest. We recommend using storage level encryption (e.g. file system encryption) to help protect sensitive data stored in PD. +* TiKV currently does not exclude encryption keys and user data from core dumps. It is adviced to disable core dumps for the TiKV process when using encryption-at-rest. This is not currently handled by TiKV itself. +* TiKV tracks encrypted data files using the absolute path of the files. As a result, once encryption is turned on for a TiKV node, the user should not change data file paths config such as `storage.data-dir`, `raftstore.raftdb-path`, `rocksdb.wal-dir` and `raftdb.wal-dir`. +* TiKV info log may contain user data for debugging purpose. Those data is not encrypted. ## TiKV Encryption-At-Rest ### Overview -TiKV currently support encrypting data using AES128, AES192 or AES256, in CTR mode. There are two types of keys used in TiKV when encryption is enabled: +TiKV currently support encrypting data using AES128, AES192 or AES256, in CTR mode. TiKV use envolope encryption, as a result, there are two types of keys used in TiKV when encryption is enabled. 1. Master key. The master key is provided by user and is used to encrypt the data keys TiKV generates. Management of master key is external to TiKV. 2. Data key. The data key is generated by TiKV and is the key actually being used to encrypt data. The data key is automatically rotated by TiKV. -The same master key can be shared by multiple instances of TiKV. The recommended way to provide a master key in production is via AWS KMS. User would need to create a CMK through AWS KMS, and then provide the CMK key id to TiKV in config file. The TiKV process would need access to the KMS CMK while it is running, which can be done by using [IAM](https://aws.amazon.com/iam/). If TiKV fail to get access to the KMS CMK, it will failed to start or restart. If TiKV loss access to the KMS CMK while it is running, data key rotation will be temporarily disabled. Please refer to AWS documentation for KMS and IAM usage. +The same master key can be shared by multiple instances of TiKV. The recommended way to provide a master key in production is via AWS KMS. Create a CMK through AWK KMS, and then provide the CMK key id to TiKV in the config file. The TiKV process needs access to the KMS CMK while it is running, which can be done by using an [IAM role](https://aws.amazon.com/iam/). If TiKV fail to get access to the KMS CMK, it will failed to start or restart. If TiKV loss access to the KMS CMK while it is running, data key rotation will be temporarily disabled. Please refer to AWS documentation for KMS and IAM usage. -Alternatively, if using custom key is desired, supplying master key via file is also supported. The file need to contain a 256 bits (or 32 bytes) key encoded as hex string. The file should end with a newline (i.e. "\n") and contain nothing else. Persistenting the key on disk, however, leaks the key, so the key file is only suitable to be stored on tempfs. +Alternatively, if using custom key is desired, supplying the master key via file is also supported. The file needs to contain a 256 bits (or 32 bytes) key encoded as hex string. The file should end with a newline (i.e. "\n") and contain nothing else. Persisting the key on disk, however, leaks the key, so the key file is only suitable to be stored on tempfs in RAM. Data keys are generated by TiKV and pass to underlying storage engine (i.e. RocksDB). All files written by RocksDB, including SST files, WAL files, and the MANIFEST file, is encrypted by the current data key. Other temporary files used by TiKV that may include user data are also encrypted using the same data key. By default, data keys are automatically rotated by TiKV every week, for which the period is configurable. On key rotation, TiKV does not rewrite all existing files to replace the key, but RocksDB compaction are expected to rewrite old data into new data files, with the most recent data key, if the cluster gets constant write workload. TiKV keeps track of the key and encryption method used to encrypt each of of files and use the information to decrypt the content on reads. @@ -39,9 +48,9 @@ data-encryption-method = aes128-ctr data-key-rotation-period = 7d ``` -Possible values for `data-encryption-method` are "aes128-ctr", "aes-192-ctr", "aes256-ctr" and "plaintext". The default value is "plaintext", which means encryption is not turned on. `data-key-rotation-period` defines how often TiKV rotate data key. Encryption can be turned on for a fresh TiKV cluster, or an existing TiKV cluster, though only data written after encryption being enabled are guaranteed to get encrypted. To disable encryption, remove `data-encryption-method` in config file, or reset it to "plaintext", and restart TiKV. To change encryption method, update `data-encryption-method` in config file and restart TiKV. +Possible values for `data-encryption-method` are "aes128-ctr", "aes192-ctr", "aes256-ctr" and "plaintext". The default value is "plaintext", which means encryption is not turned on. `data-key-rotation-period` defines how often TiKV rotate data key. Encryption can be turned on for a fresh TiKV cluster, or an existing TiKV cluster, though only data written after encryption being enabled are guaranteed to get encrypted. To disable encryption, remove `data-encryption-method` in config file, or reset it to "plaintext", and restart TiKV. To change encryption method, update `data-encryption-method` in config file and restart TiKV. -Master key has to be specified if encryption is enabled (i.e. `data-encryption-method` is not "plaintext"). To specify a AWS KMS CMK as master key, add the `encryption.master-key` section after the `encryption` section: +The master key has to be specified if encryption is enabled (i.e. `data-encryption-method` is not "plaintext"). To specify a AWS KMS CMK as master key, add the `encryption.master-key` section after the `encryption` section: ``` [security.encryption.master-key] @@ -51,7 +60,7 @@ region = "us-west-2" endpoint = "https://kms.us-west-2.amazonaws.com" ``` -The `key-id` specifies the key id for the KMS CMK. The `region` is the AWS region name for the KMS CMK. The `endpoint` is optional and don't need to be specified normally, unless you are using a AWS KMS compatible service from a non-AWS vendor. +The `key-id` specifies the key id for the KMS CMK. The `region` is the AWS region name for the KMS CMK. The `endpoint` is optional and doesn't need to be specified normally, unless you are using a AWS KMS compatible service from a non-AWS vendor. To specify a master key that's stored in a file, the master key config would look like the following: @@ -67,11 +76,11 @@ Here `path` is the path to the key file. The file must contain a 256 bits (or 16 3b5896b5be691006e0f71c3040a29495ddcad20b14aff61806940ebd780d3c62 ``` -### Rotating Master Key +### Rotating the Master Key -To rotate master key, you have to specify both of the new master key and old master key in the config, and restart TiKV. Use `encryption.master-key` to specify the new master key, and use `encryption.previous-master-key` to specify the old master key. The config format for `encryption.previous-master-key` is the same as `encryption.master-key`. On restart TiKV would need access to both of the old and new master key, but once TiKV is up and running, TiKV will only need access to the new key. It is okay to leave the `encryption.previous-master-key` config in the config file from that on. Even on restart, TiKV will only try to use the old key if it fail to decrypt existing data using the new master key. +To rotate master key, you have to specify both of the new master key and old master key in the config, and restart TiKV. Use `security.encryption.master-key` to specify the new master key, and use `security.encryption.previous-master-key` to specify the old master key. The config format for `security.encryption.previous-master-key` is the same as `encryption.master-key`. On restart TiKV must access both of the old and new master key, but once TiKV is up and running, TiKV will only need access to the new key. It is okay to leave the `encryption.previous-master-key` config in the config file from that on. Even on restart, TiKV will only try to use the old key if it fail to decrypt existing data using the new master key. -Currently we don't support online master key rotation. Rotating master key would require restarting TiKV. +Currently we don't support online master key rotation - a restart of TiKV is required. It is adviced to do a rolling restart to a running TiKV cluster serving online query. Here is an example config for rotating the KMS CMK: @@ -89,21 +98,13 @@ region = "us-west-2" ### Monitoring -To monitor encryption-at-rest, if you deploy TiKV with Grafana, you can look at the "Encryption" pannel in "TiKV-Details" dashboard. There are a few metrics to look for: +To monitor encryption-at-rest, if you deploy TiKV with Grafana, you can look at the "Encryption" panel in the "TiKV-Details" dashboard. There are a few metrics to look for: * Encryption initialized: 1 if encryption is initialized during TiKV startup, 0 otherwise. In case of master key rotation, after encryption is initialized, TiKV do not need access to the previous master key. * Encryption data keys: number of existings data keys. The number is bumped by 1 after each time data key rotation happened. Use this metrics to monitor if data key rotation works as expected. * Encrypted files: number of encrypted data files currently exists. Compare this number to existings data files in the data directory to estimate portion of data being encrypted, when turning on encryption for a previously unencrypted cluster. * Encryption meta file size: size of the encryption meta data files. -* Read/Write encryption meta duration: the extra overhead for operate on encryption metadata. - -### Caveats - -The current version of TiKV encryption has some drawbacks that need to be noted when use. We are working actively to address those issues and improvments are in the pipe for future versions. - -* When deploying a TiDB cluster, the majority of user data are stored in TiKV nodes, and those data are encrypted-at-rest when encryption is enabled. However there are small amount of user data stored in PD nodes as metadata. As of v4.0.0, PD doesn't support encryption-at-rest. As a workaround, it is recommended to use storage level encryption (e.g. file system encryption) to protect sensitive data stored in PD. -* TiKV currently does not exclude encryption keys and user data from core dump. It is highly adviced to disable core dump for TiKV process when using encryption-at-rest. This is not currently handled by TiKV itself. -* TiKV keeps track of encryption key and method used for each data files, using absolute path of the files. As a result, once encryption is turned on for a TiKV node, user should not change data file paths config such as `storage.data-dir`, `raftstore.raftdb-path`, `rocksdb.wal-dir` and `raftdb.wal-dir`. +* Read/Write encryption meta duration: the extra overhead to operate on metadata for encryption. ## BR S3 server-side encryption @@ -113,13 +114,13 @@ To enable S3 server-side encryption when backup to S3 using BR, pass `--s3.sse` ./br backup full --pd --storage "s3:///" --s3.sse aws:kms ``` -To use a custom AWS KMS CMK that you created and owned, pass `--s3.sse-kms-key-id` in addition. In this case, both of the BR process and all the TiKV nodes in the cluster would need access to the KMS CMK, and the KMS CMK need to be in the same AWS region as the S3 bucket used for store the backup. It is adviced to grant access to the KMS CMK to BR process and TiKV nodes via AWS IAM. Please refer to AWS documentation for usage of IAM. Example: +To use a custom AWS KMS CMK that you created and owned, pass `--s3.sse-kms-key-id` in addition. In this case, both the BR process and all the TiKV nodes in the cluster would need access to the KMS CMK (e.g. via AWS IAM), and the KMS CMK needs to be in the same AWS region as the S3 bucket used for store the backup. It is adviced to grant access to the KMS CMK to BR process and TiKV nodes via AWS IAM. Please refer to AWS documentation for usage of IAM. Example: ``` ./br backup full --pd --storage "s3:///" --s3.region --s3.sse aws:kms --s3.sse-kms-key-id 0987dcba-09fe-87dc-65ba-ab0987654321 ``` -On restore the backup, both `--s3.sse` and `--s3.sse-kms-key-id` should NOT be used, as S3 will figure out encryption settings by itself. The BR process and TiKV nodes in the cluster to restore the backup to would also need access to the KMS CMK, or the restore will fail. Example: +When restoring the backup, both `--s3.sse` and `--s3.sse-kms-key-id` should NOT be used. S3 will figure out encryption settings by itself. The BR process and TiKV nodes in the cluster to restore the backup to would also need access to the KMS CMK, or the restore will fail. Example: ``` ./br restore full --pd --storage "s3:/// --s3.region " From c4632bad1050ae65bdf1c927c660d487583145fa Mon Sep 17 00:00:00 2001 From: Yi Wu Date: Wed, 13 May 2020 09:19:50 +0800 Subject: [PATCH 05/30] address comments Signed-off-by: Yi Wu --- reference/security/encryption-at-rest.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/reference/security/encryption-at-rest.md b/reference/security/encryption-at-rest.md index 76f39c97606a8..a67d82f26e2bc 100644 --- a/reference/security/encryption-at-rest.md +++ b/reference/security/encryption-at-rest.md @@ -34,9 +34,9 @@ The same master key can be shared by multiple instances of TiKV. The recommended Alternatively, if using custom key is desired, supplying the master key via file is also supported. The file needs to contain a 256 bits (or 32 bytes) key encoded as hex string. The file should end with a newline (i.e. "\n") and contain nothing else. Persisting the key on disk, however, leaks the key, so the key file is only suitable to be stored on tempfs in RAM. -Data keys are generated by TiKV and pass to underlying storage engine (i.e. RocksDB). All files written by RocksDB, including SST files, WAL files, and the MANIFEST file, is encrypted by the current data key. Other temporary files used by TiKV that may include user data are also encrypted using the same data key. By default, data keys are automatically rotated by TiKV every week, for which the period is configurable. On key rotation, TiKV does not rewrite all existing files to replace the key, but RocksDB compaction are expected to rewrite old data into new data files, with the most recent data key, if the cluster gets constant write workload. TiKV keeps track of the key and encryption method used to encrypt each of of files and use the information to decrypt the content on reads. +Data keys are generated by TiKV and passed to underlying storage engine (i.e. RocksDB). All files written by RocksDB, including SST files, WAL files, and the MANIFEST file, are encrypted by the current data key. Other temporary files used by TiKV that may include user data are also encrypted using the same data key. Data keys are automatically rotated by TiKV every week by default, but the period is configurable. On key rotation, TiKV does not rewrite all existing files to replace the key, but RocksDB compaction are expected to rewrite old data into new data files, with the most recent data key, if the cluster gets constant write workload. TiKV keeps track of the key and encryption method used to encrypt each of of files and use the information to decrypt the content on reads. -Regardless of data encryption method, data keys are encrypted using AES256 in GCM mode for additional authentication. This is why when passing master key from file the key has to be of 256 bits. +Regardless of data encryption method, data keys are encrypted using AES256 in GCM mode for additional authentication. This required the master key to be 256 bits (32 bytes), when passing from file instead of KMS. ### Configuring Encryption @@ -48,7 +48,7 @@ data-encryption-method = aes128-ctr data-key-rotation-period = 7d ``` -Possible values for `data-encryption-method` are "aes128-ctr", "aes192-ctr", "aes256-ctr" and "plaintext". The default value is "plaintext", which means encryption is not turned on. `data-key-rotation-period` defines how often TiKV rotate data key. Encryption can be turned on for a fresh TiKV cluster, or an existing TiKV cluster, though only data written after encryption being enabled are guaranteed to get encrypted. To disable encryption, remove `data-encryption-method` in config file, or reset it to "plaintext", and restart TiKV. To change encryption method, update `data-encryption-method` in config file and restart TiKV. +Possible values for `data-encryption-method` are "aes128-ctr", "aes192-ctr", "aes256-ctr" and "plaintext". The default value is "plaintext", which means encryption is not turned on. `data-key-rotation-period` defines how often TiKV rotates the data key. Encryption can be turned on for a fresh TiKV cluster, or an existing TiKV cluster, though only data written after encryption is enabled is guaranteed to be encrypted. To disable encryption, remove `data-encryption-method` in config file, or reset it to "plaintext", and restart TiKV. To change encryption method, update `data-encryption-method` in config file and restart TiKV. The master key has to be specified if encryption is enabled (i.e. `data-encryption-method` is not "plaintext"). To specify a AWS KMS CMK as master key, add the `encryption.master-key` section after the `encryption` section: From a77d67c0842b375657998dcc58a4328e437ed684 Mon Sep 17 00:00:00 2001 From: Yi Wu Date: Wed, 13 May 2020 09:53:20 +0800 Subject: [PATCH 06/30] fix lint Signed-off-by: Yi Wu --- reference/security/encryption-at-rest.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/reference/security/encryption-at-rest.md b/reference/security/encryption-at-rest.md index a67d82f26e2bc..aae2a345c426a 100644 --- a/reference/security/encryption-at-rest.md +++ b/reference/security/encryption-at-rest.md @@ -12,7 +12,7 @@ TiKV supports encryption-at-rest starting from v4.0.0. The feature allows TiKV t Also from v4.0.0, BR supports S3 server-side encryption (SSE) when backing up to S3. A customer owned AWS KMS key can also be used together with S3 server-side encrytion. -### Caveats +## Caveats The current version of TiKV encryption has some drawbacks that we are working actively to address in the future versions. From dc8b4c375f95106e2267cfd6d8bb48eceb8d26a0 Mon Sep 17 00:00:00 2001 From: Yi Wu Date: Wed, 13 May 2020 11:45:54 +0800 Subject: [PATCH 07/30] address comments Signed-off-by: Yi Wu --- reference/security/encryption-at-rest.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/reference/security/encryption-at-rest.md b/reference/security/encryption-at-rest.md index aae2a345c426a..b94e69cc72bbc 100644 --- a/reference/security/encryption-at-rest.md +++ b/reference/security/encryption-at-rest.md @@ -6,9 +6,9 @@ category: reference # Encryption-At-Rest for TiKV New in v4.0.0 -Encryption-at-rest means that data is encrypted when it is stored. For databases, this feature is also referred to as TDE (transparent data encryption). This is opposed to encryption in flight (TLS) or encryption in use (rarely used). Different things could be doing encryption-at-rest (SSD drive, file system, cloud vendor, etc), but by having TiKV do the encryption before storage this helps ensure that attackers must authenticate with the database to gain access to data. For example, when an attacker gains access to the physical machine, data cannot be accessed data by copying files on disk. +Encryption-at-rest means that data is encrypted when it is stored. For databases, this feature is also referred to as TDE (transparent data encryption). This is opposed to encryption in flight (TLS) or encryption in use (rarely used). Different things could be doing encryption-at-rest (SSD drive, file system, cloud vendor, etc), but by having TiKV do the encryption before storage this helps ensure that attackers must authenticate with the database to gain access to data. For example, when an attacker gains access to the physical machine, data cannot be accessed by copying files on disk. -TiKV supports encryption-at-rest starting from v4.0.0. The feature allows TiKV to transparently encrypt data files using [AES](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard) in [CTR](https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation) mode. To enable encryption-at-rest, an encryption key must be provided by user and this key is called master key. The master key can be provided via AWS KMS (recommended), or specifying a key stored as plaintext in a file. TiKV automatically rotate data keys that it used to encrypt actual data files. Manually rotating the master key can be done occassionally. Note that encryption-at-rest only encrypts data at rest (i.e. on disk) and not while data is transferred over network. It is advices to use TLS together with encryption-at-rest. +TiKV supports encryption-at-rest starting from v4.0.0. The feature allows TiKV to transparently encrypt data files using [AES](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard) in [CTR](https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation) mode. To enable encryption-at-rest, an encryption key must be provided by user and this key is called master key. The master key can be provided via AWS KMS (recommended), or specifying a key stored as plaintext in a file. TiKV automatically rotates data keys that it used to encrypt actual data files. Manually rotating the master key can be done occassionally. Note that encryption-at-rest only encrypts data at rest (i.e. on disk) and not while data is transferred over network. It is adviced to use TLS together with encryption-at-rest. Also from v4.0.0, BR supports S3 server-side encryption (SSE) when backing up to S3. A customer owned AWS KMS key can also be used together with S3 server-side encrytion. @@ -19,7 +19,7 @@ The current version of TiKV encryption has some drawbacks that we are working ac * When deploying a TiDB cluster, the majority of a user's data is stored in TiKV nodes, and that data will be encrypted when encryption is enabled. However there is a small amount of user data stored in PD nodes as metadata (e.g secondary index keys used as TiKV region boundaries). As of v4.0.0, PD doesn't support encryption-at-rest. We recommend using storage level encryption (e.g. file system encryption) to help protect sensitive data stored in PD. * TiKV currently does not exclude encryption keys and user data from core dumps. It is adviced to disable core dumps for the TiKV process when using encryption-at-rest. This is not currently handled by TiKV itself. * TiKV tracks encrypted data files using the absolute path of the files. As a result, once encryption is turned on for a TiKV node, the user should not change data file paths config such as `storage.data-dir`, `raftstore.raftdb-path`, `rocksdb.wal-dir` and `raftdb.wal-dir`. -* TiKV info log may contain user data for debugging purpose. Those data is not encrypted. +* TiKV info log contains user data for debugging purposes. The info log and this data in it are not encrypted. ## TiKV Encryption-At-Rest From b6efa175b2fd16156bbb22cc93a44b9a24bddf00 Mon Sep 17 00:00:00 2001 From: Yi Wu Date: Thu, 14 May 2020 07:01:59 +0800 Subject: [PATCH 08/30] update caveat for TiFlash Signed-off-by: Yi Wu --- reference/security/encryption-at-rest.md | 1 + 1 file changed, 1 insertion(+) diff --git a/reference/security/encryption-at-rest.md b/reference/security/encryption-at-rest.md index b94e69cc72bbc..16fdb6ce77143 100644 --- a/reference/security/encryption-at-rest.md +++ b/reference/security/encryption-at-rest.md @@ -17,6 +17,7 @@ Also from v4.0.0, BR supports S3 server-side encryption (SSE) when backing up to The current version of TiKV encryption has some drawbacks that we are working actively to address in the future versions. * When deploying a TiDB cluster, the majority of a user's data is stored in TiKV nodes, and that data will be encrypted when encryption is enabled. However there is a small amount of user data stored in PD nodes as metadata (e.g secondary index keys used as TiKV region boundaries). As of v4.0.0, PD doesn't support encryption-at-rest. We recommend using storage level encryption (e.g. file system encryption) to help protect sensitive data stored in PD. +* As of v4.0.0, TiFlash doesn't support encryption-at-rest. When deploying TiKV with TiFlash, data stored in TiFlash is not encrypted. * TiKV currently does not exclude encryption keys and user data from core dumps. It is adviced to disable core dumps for the TiKV process when using encryption-at-rest. This is not currently handled by TiKV itself. * TiKV tracks encrypted data files using the absolute path of the files. As a result, once encryption is turned on for a TiKV node, the user should not change data file paths config such as `storage.data-dir`, `raftstore.raftdb-path`, `rocksdb.wal-dir` and `raftdb.wal-dir`. * TiKV info log contains user data for debugging purposes. The info log and this data in it are not encrypted. From 6cca2973209c9082abec567b31b0aaea4c475b72 Mon Sep 17 00:00:00 2001 From: yiwu-arbug Date: Wed, 13 May 2020 22:03:37 -0700 Subject: [PATCH 09/30] Update reference/security/encryption-at-rest.md Co-authored-by: Ran --- reference/security/encryption-at-rest.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/reference/security/encryption-at-rest.md b/reference/security/encryption-at-rest.md index 16fdb6ce77143..a82d4a14c82ae 100644 --- a/reference/security/encryption-at-rest.md +++ b/reference/security/encryption-at-rest.md @@ -1,5 +1,5 @@ --- -title: Encryption-At-Rest +title: Encryption-At-Rest for TiKV summary: Learn how to enable encryption-at-rest to protect sensitive data. category: reference --- From ff906bbcf7ebb1b4f57330cae8c8edd9ec042b32 Mon Sep 17 00:00:00 2001 From: yiwu-arbug Date: Wed, 13 May 2020 22:04:32 -0700 Subject: [PATCH 10/30] Update reference/security/encryption-at-rest.md Co-authored-by: Ran --- reference/security/encryption-at-rest.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/reference/security/encryption-at-rest.md b/reference/security/encryption-at-rest.md index a82d4a14c82ae..ddedc194929d8 100644 --- a/reference/security/encryption-at-rest.md +++ b/reference/security/encryption-at-rest.md @@ -16,7 +16,7 @@ Also from v4.0.0, BR supports S3 server-side encryption (SSE) when backing up to The current version of TiKV encryption has some drawbacks that we are working actively to address in the future versions. -* When deploying a TiDB cluster, the majority of a user's data is stored in TiKV nodes, and that data will be encrypted when encryption is enabled. However there is a small amount of user data stored in PD nodes as metadata (e.g secondary index keys used as TiKV region boundaries). As of v4.0.0, PD doesn't support encryption-at-rest. We recommend using storage level encryption (e.g. file system encryption) to help protect sensitive data stored in PD. +* When a TiDB cluster is deployed, the majority of user data is stored in TiKV nodes, and that data will be encrypted when encryption is enabled. However, there is a small amount of user data stored in PD nodes as metadata (for example, secondary index keys used as TiKV region boundaries). As of v4.0.0, PD doesn't support encryption-at-rest. It is recommended to use storage-level encryption (for example, file system encryption) to help protect sensitive data stored in PD. * As of v4.0.0, TiFlash doesn't support encryption-at-rest. When deploying TiKV with TiFlash, data stored in TiFlash is not encrypted. * TiKV currently does not exclude encryption keys and user data from core dumps. It is adviced to disable core dumps for the TiKV process when using encryption-at-rest. This is not currently handled by TiKV itself. * TiKV tracks encrypted data files using the absolute path of the files. As a result, once encryption is turned on for a TiKV node, the user should not change data file paths config such as `storage.data-dir`, `raftstore.raftdb-path`, `rocksdb.wal-dir` and `raftdb.wal-dir`. From ca4d9f27f454da0a7288857d1f191fe3783f4c81 Mon Sep 17 00:00:00 2001 From: yiwu-arbug Date: Wed, 13 May 2020 22:05:14 -0700 Subject: [PATCH 11/30] Update reference/security/encryption-at-rest.md Co-authored-by: Ran --- reference/security/encryption-at-rest.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/reference/security/encryption-at-rest.md b/reference/security/encryption-at-rest.md index ddedc194929d8..fdf9bc5bb49f7 100644 --- a/reference/security/encryption-at-rest.md +++ b/reference/security/encryption-at-rest.md @@ -26,7 +26,7 @@ The current version of TiKV encryption has some drawbacks that we are working ac ### Overview -TiKV currently support encrypting data using AES128, AES192 or AES256, in CTR mode. TiKV use envolope encryption, as a result, there are two types of keys used in TiKV when encryption is enabled. +TiKV currently supports encrypting data using AES128, AES192 or AES256, in CTR mode. TiKV uses envolope encryption. As a result, there are two types of keys used in TiKV when encryption is enabled. 1. Master key. The master key is provided by user and is used to encrypt the data keys TiKV generates. Management of master key is external to TiKV. 2. Data key. The data key is generated by TiKV and is the key actually being used to encrypt data. The data key is automatically rotated by TiKV. From 676c501d5c3b879d453adf720c6d2429c03be4fb Mon Sep 17 00:00:00 2001 From: yiwu-arbug Date: Wed, 13 May 2020 22:05:32 -0700 Subject: [PATCH 12/30] Update reference/security/encryption-at-rest.md Co-authored-by: Ran --- reference/security/encryption-at-rest.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/reference/security/encryption-at-rest.md b/reference/security/encryption-at-rest.md index fdf9bc5bb49f7..730bbc9616372 100644 --- a/reference/security/encryption-at-rest.md +++ b/reference/security/encryption-at-rest.md @@ -29,7 +29,7 @@ The current version of TiKV encryption has some drawbacks that we are working ac TiKV currently supports encrypting data using AES128, AES192 or AES256, in CTR mode. TiKV uses envolope encryption. As a result, there are two types of keys used in TiKV when encryption is enabled. 1. Master key. The master key is provided by user and is used to encrypt the data keys TiKV generates. Management of master key is external to TiKV. -2. Data key. The data key is generated by TiKV and is the key actually being used to encrypt data. The data key is automatically rotated by TiKV. +2. Data key. The data key is generated by TiKV and is the key actually used to encrypt data. The data key is automatically rotated by TiKV. The same master key can be shared by multiple instances of TiKV. The recommended way to provide a master key in production is via AWS KMS. Create a CMK through AWK KMS, and then provide the CMK key id to TiKV in the config file. The TiKV process needs access to the KMS CMK while it is running, which can be done by using an [IAM role](https://aws.amazon.com/iam/). If TiKV fail to get access to the KMS CMK, it will failed to start or restart. If TiKV loss access to the KMS CMK while it is running, data key rotation will be temporarily disabled. Please refer to AWS documentation for KMS and IAM usage. From 786f97ecba256d79042fdfcbfc3f9677bae9672d Mon Sep 17 00:00:00 2001 From: yiwu-arbug Date: Wed, 13 May 2020 22:06:04 -0700 Subject: [PATCH 13/30] Update reference/security/encryption-at-rest.md Co-authored-by: Ran --- reference/security/encryption-at-rest.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/reference/security/encryption-at-rest.md b/reference/security/encryption-at-rest.md index 730bbc9616372..461bc66dfd10e 100644 --- a/reference/security/encryption-at-rest.md +++ b/reference/security/encryption-at-rest.md @@ -31,7 +31,7 @@ TiKV currently supports encrypting data using AES128, AES192 or AES256, in CTR m 1. Master key. The master key is provided by user and is used to encrypt the data keys TiKV generates. Management of master key is external to TiKV. 2. Data key. The data key is generated by TiKV and is the key actually used to encrypt data. The data key is automatically rotated by TiKV. -The same master key can be shared by multiple instances of TiKV. The recommended way to provide a master key in production is via AWS KMS. Create a CMK through AWK KMS, and then provide the CMK key id to TiKV in the config file. The TiKV process needs access to the KMS CMK while it is running, which can be done by using an [IAM role](https://aws.amazon.com/iam/). If TiKV fail to get access to the KMS CMK, it will failed to start or restart. If TiKV loss access to the KMS CMK while it is running, data key rotation will be temporarily disabled. Please refer to AWS documentation for KMS and IAM usage. +The same master key can be shared by multiple instances of TiKV. The recommended way to provide a master key in production is via AWS KMS. Create a CMK through AWS KMS, and then provide the CMK key ID to TiKV in the config file. The TiKV process needs access to the KMS CMK while it is running, which can be done by using an [IAM role](https://aws.amazon.com/iam/). If TiKV fails to get access to the KMS CMK, it will fail to start or restart. If TiKV loses access to the KMS CMK while it is running, data key rotation will be temporarily disabled. Please refer to AWS documentation for KMS and IAM usage. Alternatively, if using custom key is desired, supplying the master key via file is also supported. The file needs to contain a 256 bits (or 32 bytes) key encoded as hex string. The file should end with a newline (i.e. "\n") and contain nothing else. Persisting the key on disk, however, leaks the key, so the key file is only suitable to be stored on tempfs in RAM. From c130b99e37684158a2720ba4cec74e5aa4750a62 Mon Sep 17 00:00:00 2001 From: yiwu-arbug Date: Wed, 13 May 2020 22:07:16 -0700 Subject: [PATCH 14/30] Update reference/security/encryption-at-rest.md Co-authored-by: Ran --- reference/security/encryption-at-rest.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/reference/security/encryption-at-rest.md b/reference/security/encryption-at-rest.md index 461bc66dfd10e..0866253cb9677 100644 --- a/reference/security/encryption-at-rest.md +++ b/reference/security/encryption-at-rest.md @@ -35,7 +35,7 @@ The same master key can be shared by multiple instances of TiKV. The recommended Alternatively, if using custom key is desired, supplying the master key via file is also supported. The file needs to contain a 256 bits (or 32 bytes) key encoded as hex string. The file should end with a newline (i.e. "\n") and contain nothing else. Persisting the key on disk, however, leaks the key, so the key file is only suitable to be stored on tempfs in RAM. -Data keys are generated by TiKV and passed to underlying storage engine (i.e. RocksDB). All files written by RocksDB, including SST files, WAL files, and the MANIFEST file, are encrypted by the current data key. Other temporary files used by TiKV that may include user data are also encrypted using the same data key. Data keys are automatically rotated by TiKV every week by default, but the period is configurable. On key rotation, TiKV does not rewrite all existing files to replace the key, but RocksDB compaction are expected to rewrite old data into new data files, with the most recent data key, if the cluster gets constant write workload. TiKV keeps track of the key and encryption method used to encrypt each of of files and use the information to decrypt the content on reads. +Data keys are generated by TiKV and passed to the underlying storage engine (i.e. RocksDB). All files written by RocksDB, including SST files, WAL files, and the MANIFEST file, are encrypted by the current data key. Other temporary files used by TiKV that may include user data are also encrypted using the same data key. Data keys are automatically rotated by TiKV every week by default, but the period is configurable. On key rotation, TiKV does not rewrite all existing files to replace the key, but RocksDB compaction are expected to rewrite old data into new data files, with the most recent data key, if the cluster gets constant write workload. TiKV keeps track of the key and encryption method used to encrypt each of the files and use the information to decrypt the content on reads. Regardless of data encryption method, data keys are encrypted using AES256 in GCM mode for additional authentication. This required the master key to be 256 bits (32 bytes), when passing from file instead of KMS. From efd0962699138e5277ac2fec7085000c67079346 Mon Sep 17 00:00:00 2001 From: yikeke Date: Thu, 14 May 2020 17:32:40 +0800 Subject: [PATCH 15/30] relocate the file --- reference/security/encryption-at-rest.md => encryption-at-rest.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename reference/security/encryption-at-rest.md => encryption-at-rest.md (100%) diff --git a/reference/security/encryption-at-rest.md b/encryption-at-rest.md similarity index 100% rename from reference/security/encryption-at-rest.md rename to encryption-at-rest.md From a38477e39552b1cf7e920ca97f840b774bc348a7 Mon Sep 17 00:00:00 2001 From: yiwu-arbug Date: Mon, 18 May 2020 14:35:22 -0700 Subject: [PATCH 16/30] Update encryption-at-rest.md Co-authored-by: Ran --- encryption-at-rest.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/encryption-at-rest.md b/encryption-at-rest.md index 0866253cb9677..58bde2cc7f36d 100644 --- a/encryption-at-rest.md +++ b/encryption-at-rest.md @@ -16,7 +16,7 @@ Also from v4.0.0, BR supports S3 server-side encryption (SSE) when backing up to The current version of TiKV encryption has some drawbacks that we are working actively to address in the future versions. -* When a TiDB cluster is deployed, the majority of user data is stored in TiKV nodes, and that data will be encrypted when encryption is enabled. However, there is a small amount of user data stored in PD nodes as metadata (for example, secondary index keys used as TiKV region boundaries). As of v4.0.0, PD doesn't support encryption-at-rest. It is recommended to use storage-level encryption (for example, file system encryption) to help protect sensitive data stored in PD. +* When a TiDB cluster is deployed, the majority of user data is stored in TiKV nodes, and that data will be encrypted when encryption is enabled. However, a small amount of user data is stored in PD nodes as metadata (for example, secondary index keys used as TiKV region boundaries). As of v4.0.0, PD doesn't support encryption-at-rest. It is recommended to use storage-level encryption (for example, file system encryption) to help protect sensitive data stored in PD. * As of v4.0.0, TiFlash doesn't support encryption-at-rest. When deploying TiKV with TiFlash, data stored in TiFlash is not encrypted. * TiKV currently does not exclude encryption keys and user data from core dumps. It is adviced to disable core dumps for the TiKV process when using encryption-at-rest. This is not currently handled by TiKV itself. * TiKV tracks encrypted data files using the absolute path of the files. As a result, once encryption is turned on for a TiKV node, the user should not change data file paths config such as `storage.data-dir`, `raftstore.raftdb-path`, `rocksdb.wal-dir` and `raftdb.wal-dir`. From be34676d5c9f2a82577525383c64c6989c602442 Mon Sep 17 00:00:00 2001 From: yiwu-arbug Date: Mon, 18 May 2020 14:35:47 -0700 Subject: [PATCH 17/30] Update encryption-at-rest.md Co-authored-by: Ran --- encryption-at-rest.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/encryption-at-rest.md b/encryption-at-rest.md index 58bde2cc7f36d..ba8df289fa441 100644 --- a/encryption-at-rest.md +++ b/encryption-at-rest.md @@ -8,7 +8,7 @@ category: reference Encryption-at-rest means that data is encrypted when it is stored. For databases, this feature is also referred to as TDE (transparent data encryption). This is opposed to encryption in flight (TLS) or encryption in use (rarely used). Different things could be doing encryption-at-rest (SSD drive, file system, cloud vendor, etc), but by having TiKV do the encryption before storage this helps ensure that attackers must authenticate with the database to gain access to data. For example, when an attacker gains access to the physical machine, data cannot be accessed by copying files on disk. -TiKV supports encryption-at-rest starting from v4.0.0. The feature allows TiKV to transparently encrypt data files using [AES](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard) in [CTR](https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation) mode. To enable encryption-at-rest, an encryption key must be provided by user and this key is called master key. The master key can be provided via AWS KMS (recommended), or specifying a key stored as plaintext in a file. TiKV automatically rotates data keys that it used to encrypt actual data files. Manually rotating the master key can be done occassionally. Note that encryption-at-rest only encrypts data at rest (i.e. on disk) and not while data is transferred over network. It is adviced to use TLS together with encryption-at-rest. +TiKV supports encryption-at-rest starting from v4.0.0. The feature allows TiKV to transparently encrypt data files using [AES](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard) in [CTR](https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation) mode. To enable encryption-at-rest, an encryption key must be provided by user and this key is called master key. The master key can be provided via AWS KMS (recommended), or specifying a key stored as plaintext in a file. TiKV automatically rotates data keys that it used to encrypt actual data files. Manually rotating the master key can be done occassionally. Note that encryption-at-rest only encrypts data at rest (i.e. on disk) and not while data is transferred over network. It is advised to use TLS together with encryption-at-rest. Also from v4.0.0, BR supports S3 server-side encryption (SSE) when backing up to S3. A customer owned AWS KMS key can also be used together with S3 server-side encrytion. From 4e031d9d661bb2df8290566aaf5ae03d560866d7 Mon Sep 17 00:00:00 2001 From: yiwu-arbug Date: Mon, 18 May 2020 14:35:59 -0700 Subject: [PATCH 18/30] Update encryption-at-rest.md Co-authored-by: Ran --- encryption-at-rest.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/encryption-at-rest.md b/encryption-at-rest.md index ba8df289fa441..d00fbf43a5aae 100644 --- a/encryption-at-rest.md +++ b/encryption-at-rest.md @@ -18,7 +18,7 @@ The current version of TiKV encryption has some drawbacks that we are working ac * When a TiDB cluster is deployed, the majority of user data is stored in TiKV nodes, and that data will be encrypted when encryption is enabled. However, a small amount of user data is stored in PD nodes as metadata (for example, secondary index keys used as TiKV region boundaries). As of v4.0.0, PD doesn't support encryption-at-rest. It is recommended to use storage-level encryption (for example, file system encryption) to help protect sensitive data stored in PD. * As of v4.0.0, TiFlash doesn't support encryption-at-rest. When deploying TiKV with TiFlash, data stored in TiFlash is not encrypted. -* TiKV currently does not exclude encryption keys and user data from core dumps. It is adviced to disable core dumps for the TiKV process when using encryption-at-rest. This is not currently handled by TiKV itself. +* TiKV currently does not exclude encryption keys and user data from core dumps. It is advised to disable core dumps for the TiKV process when using encryption-at-rest. This is not currently handled by TiKV itself. * TiKV tracks encrypted data files using the absolute path of the files. As a result, once encryption is turned on for a TiKV node, the user should not change data file paths config such as `storage.data-dir`, `raftstore.raftdb-path`, `rocksdb.wal-dir` and `raftdb.wal-dir`. * TiKV info log contains user data for debugging purposes. The info log and this data in it are not encrypted. From 908d6b9ca3c79ebd0ba4b86da38b8fa213f5bb65 Mon Sep 17 00:00:00 2001 From: yiwu-arbug Date: Mon, 18 May 2020 14:38:06 -0700 Subject: [PATCH 19/30] Update encryption-at-rest.md Co-authored-by: Ran --- encryption-at-rest.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/encryption-at-rest.md b/encryption-at-rest.md index d00fbf43a5aae..b3b65ac97aaae 100644 --- a/encryption-at-rest.md +++ b/encryption-at-rest.md @@ -26,7 +26,7 @@ The current version of TiKV encryption has some drawbacks that we are working ac ### Overview -TiKV currently supports encrypting data using AES128, AES192 or AES256, in CTR mode. TiKV uses envolope encryption. As a result, there are two types of keys used in TiKV when encryption is enabled. +TiKV currently supports encrypting data using AES128, AES192 or AES256, in CTR mode. TiKV uses envelope encryption. As a result, two types of keys are used in TiKV when encryption is enabled. 1. Master key. The master key is provided by user and is used to encrypt the data keys TiKV generates. Management of master key is external to TiKV. 2. Data key. The data key is generated by TiKV and is the key actually used to encrypt data. The data key is automatically rotated by TiKV. From ad7aa9783cdca4317916da96e5d1d995008ef116 Mon Sep 17 00:00:00 2001 From: yiwu-arbug Date: Mon, 18 May 2020 14:38:22 -0700 Subject: [PATCH 20/30] Update encryption-at-rest.md Co-authored-by: Ran --- encryption-at-rest.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/encryption-at-rest.md b/encryption-at-rest.md index b3b65ac97aaae..8688f4adccc13 100644 --- a/encryption-at-rest.md +++ b/encryption-at-rest.md @@ -28,8 +28,8 @@ The current version of TiKV encryption has some drawbacks that we are working ac TiKV currently supports encrypting data using AES128, AES192 or AES256, in CTR mode. TiKV uses envelope encryption. As a result, two types of keys are used in TiKV when encryption is enabled. -1. Master key. The master key is provided by user and is used to encrypt the data keys TiKV generates. Management of master key is external to TiKV. -2. Data key. The data key is generated by TiKV and is the key actually used to encrypt data. The data key is automatically rotated by TiKV. +* Master key. The master key is provided by user and is used to encrypt the data keys TiKV generates. Management of master key is external to TiKV. +* Data key. The data key is generated by TiKV and is the key actually used to encrypt data. The data key is automatically rotated by TiKV. The same master key can be shared by multiple instances of TiKV. The recommended way to provide a master key in production is via AWS KMS. Create a CMK through AWS KMS, and then provide the CMK key ID to TiKV in the config file. The TiKV process needs access to the KMS CMK while it is running, which can be done by using an [IAM role](https://aws.amazon.com/iam/). If TiKV fails to get access to the KMS CMK, it will fail to start or restart. If TiKV loses access to the KMS CMK while it is running, data key rotation will be temporarily disabled. Please refer to AWS documentation for KMS and IAM usage. From 528b681c5310ea1e8fdff7e1deeadd5553d17e53 Mon Sep 17 00:00:00 2001 From: yiwu-arbug Date: Mon, 18 May 2020 14:38:52 -0700 Subject: [PATCH 21/30] Update encryption-at-rest.md Co-authored-by: Ran --- encryption-at-rest.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/encryption-at-rest.md b/encryption-at-rest.md index 8688f4adccc13..8e4ac56313607 100644 --- a/encryption-at-rest.md +++ b/encryption-at-rest.md @@ -43,6 +43,8 @@ Regardless of data encryption method, data keys are encrypted using AES256 in GC To enable encryption, you can add the encryption section in TiKV's config file: +{{< copyable "" >}} + ``` [security.encryption] data-encryption-method = aes128-ctr From df4361f69763d3efa5feefbd8f1cc296c4f8591b Mon Sep 17 00:00:00 2001 From: yiwu-arbug Date: Mon, 18 May 2020 14:39:06 -0700 Subject: [PATCH 22/30] Update encryption-at-rest.md Co-authored-by: Ran --- encryption-at-rest.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/encryption-at-rest.md b/encryption-at-rest.md index 8e4ac56313607..28760cc2f1544 100644 --- a/encryption-at-rest.md +++ b/encryption-at-rest.md @@ -55,6 +55,8 @@ Possible values for `data-encryption-method` are "aes128-ctr", "aes192-ctr", "ae The master key has to be specified if encryption is enabled (i.e. `data-encryption-method` is not "plaintext"). To specify a AWS KMS CMK as master key, add the `encryption.master-key` section after the `encryption` section: +{{< copyable "" >}} + ``` [security.encryption.master-key] type = "kms" From 522667bc9e46d75d6a0064baf574185bc97d662e Mon Sep 17 00:00:00 2001 From: yiwu-arbug Date: Mon, 18 May 2020 14:40:22 -0700 Subject: [PATCH 23/30] Update encryption-at-rest.md Co-authored-by: Ran --- encryption-at-rest.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/encryption-at-rest.md b/encryption-at-rest.md index 28760cc2f1544..defd1dd922e57 100644 --- a/encryption-at-rest.md +++ b/encryption-at-rest.md @@ -85,7 +85,7 @@ Here `path` is the path to the key file. The file must contain a 256 bits (or 16 To rotate master key, you have to specify both of the new master key and old master key in the config, and restart TiKV. Use `security.encryption.master-key` to specify the new master key, and use `security.encryption.previous-master-key` to specify the old master key. The config format for `security.encryption.previous-master-key` is the same as `encryption.master-key`. On restart TiKV must access both of the old and new master key, but once TiKV is up and running, TiKV will only need access to the new key. It is okay to leave the `encryption.previous-master-key` config in the config file from that on. Even on restart, TiKV will only try to use the old key if it fail to decrypt existing data using the new master key. -Currently we don't support online master key rotation - a restart of TiKV is required. It is adviced to do a rolling restart to a running TiKV cluster serving online query. +Currently online master key rotation is not supported, so you need to restart TiKV. It is advised to do a rolling restart to a running TiKV cluster serving online query. Here is an example config for rotating the KMS CMK: From 7d578eedc531092688f2d6da9aeb7c6a77d490d3 Mon Sep 17 00:00:00 2001 From: yiwu-arbug Date: Mon, 18 May 2020 14:40:38 -0700 Subject: [PATCH 24/30] Update encryption-at-rest.md Co-authored-by: Ran --- encryption-at-rest.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/encryption-at-rest.md b/encryption-at-rest.md index defd1dd922e57..001e6aec7c1ae 100644 --- a/encryption-at-rest.md +++ b/encryption-at-rest.md @@ -103,7 +103,7 @@ region = "us-west-2" ### Monitoring -To monitor encryption-at-rest, if you deploy TiKV with Grafana, you can look at the "Encryption" panel in the "TiKV-Details" dashboard. There are a few metrics to look for: +To monitor encryption-at-rest, if you deploy TiKV with Grafana, you can look at the **Encryption** panel in the **TiKV-Details** dashboard. There are a few metrics to look for: * Encryption initialized: 1 if encryption is initialized during TiKV startup, 0 otherwise. In case of master key rotation, after encryption is initialized, TiKV do not need access to the previous master key. * Encryption data keys: number of existings data keys. The number is bumped by 1 after each time data key rotation happened. Use this metrics to monitor if data key rotation works as expected. From ef7da349600968134b9baf141099fd266273a588 Mon Sep 17 00:00:00 2001 From: yiwu-arbug Date: Mon, 18 May 2020 14:42:12 -0700 Subject: [PATCH 25/30] Update encryption-at-rest.md Co-authored-by: Ran --- encryption-at-rest.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/encryption-at-rest.md b/encryption-at-rest.md index 001e6aec7c1ae..c2d9ed85c76da 100644 --- a/encryption-at-rest.md +++ b/encryption-at-rest.md @@ -31,7 +31,7 @@ TiKV currently supports encrypting data using AES128, AES192 or AES256, in CTR m * Master key. The master key is provided by user and is used to encrypt the data keys TiKV generates. Management of master key is external to TiKV. * Data key. The data key is generated by TiKV and is the key actually used to encrypt data. The data key is automatically rotated by TiKV. -The same master key can be shared by multiple instances of TiKV. The recommended way to provide a master key in production is via AWS KMS. Create a CMK through AWS KMS, and then provide the CMK key ID to TiKV in the config file. The TiKV process needs access to the KMS CMK while it is running, which can be done by using an [IAM role](https://aws.amazon.com/iam/). If TiKV fails to get access to the KMS CMK, it will fail to start or restart. If TiKV loses access to the KMS CMK while it is running, data key rotation will be temporarily disabled. Please refer to AWS documentation for KMS and IAM usage. +The same master key can be shared by multiple instances of TiKV. The recommended way to provide a master key in production is via AWS KMS. Create a customer master key (CMK) through AWS KMS, and then provide the CMK key ID to TiKV in the config file. The TiKV process needs access to the KMS CMK while it is running, which can be done by using an [IAM role](https://aws.amazon.com/iam/). If TiKV fails to get access to the KMS CMK, it will fail to start or restart. If TiKV loses access to the KMS CMK while it is running, data key rotation will be temporarily disabled. Refer to AWS documentation for [KMS](https://docs.aws.amazon.com/kms/index.html) and [IAM](https://docs.aws.amazon.com/IAM/latest/UserGuide/introduction.html) usage. Alternatively, if using custom key is desired, supplying the master key via file is also supported. The file needs to contain a 256 bits (or 32 bytes) key encoded as hex string. The file should end with a newline (i.e. "\n") and contain nothing else. Persisting the key on disk, however, leaks the key, so the key file is only suitable to be stored on tempfs in RAM. From a768bedaf37b2ac64aef3230e69d3419945ee812 Mon Sep 17 00:00:00 2001 From: yiwu-arbug Date: Mon, 18 May 2020 14:44:00 -0700 Subject: [PATCH 26/30] Update encryption-at-rest.md Co-authored-by: Ran --- encryption-at-rest.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/encryption-at-rest.md b/encryption-at-rest.md index c2d9ed85c76da..504c2e6b122e9 100644 --- a/encryption-at-rest.md +++ b/encryption-at-rest.md @@ -51,7 +51,7 @@ data-encryption-method = aes128-ctr data-key-rotation-period = 7d ``` -Possible values for `data-encryption-method` are "aes128-ctr", "aes192-ctr", "aes256-ctr" and "plaintext". The default value is "plaintext", which means encryption is not turned on. `data-key-rotation-period` defines how often TiKV rotates the data key. Encryption can be turned on for a fresh TiKV cluster, or an existing TiKV cluster, though only data written after encryption is enabled is guaranteed to be encrypted. To disable encryption, remove `data-encryption-method` in config file, or reset it to "plaintext", and restart TiKV. To change encryption method, update `data-encryption-method` in config file and restart TiKV. +Possible values for `data-encryption-method` are "aes128-ctr", "aes192-ctr", "aes256-ctr" and "plaintext". The default value is "plaintext", which means encryption is not turned on. `data-key-rotation-period` defines how often TiKV rotates the data key. Encryption can be turned on for a fresh TiKV cluster, or an existing TiKV cluster, though only data written after encryption is enabled is guaranteed to be encrypted. To disable encryption, remove `data-encryption-method` in the config file, or reset it to "plaintext", and restart TiKV. To change encryption method, update `data-encryption-method` in the config file and restart TiKV. The master key has to be specified if encryption is enabled (i.e. `data-encryption-method` is not "plaintext"). To specify a AWS KMS CMK as master key, add the `encryption.master-key` section after the `encryption` section: From 6ba90a05391b4a8a59953ea4bfadc7f5890ae096 Mon Sep 17 00:00:00 2001 From: yiwu-arbug Date: Mon, 18 May 2020 14:44:35 -0700 Subject: [PATCH 27/30] Update encryption-at-rest.md Co-authored-by: Ran --- encryption-at-rest.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/encryption-at-rest.md b/encryption-at-rest.md index 504c2e6b122e9..1fd3f5f3fc599 100644 --- a/encryption-at-rest.md +++ b/encryption-at-rest.md @@ -83,7 +83,7 @@ Here `path` is the path to the key file. The file must contain a 256 bits (or 16 ### Rotating the Master Key -To rotate master key, you have to specify both of the new master key and old master key in the config, and restart TiKV. Use `security.encryption.master-key` to specify the new master key, and use `security.encryption.previous-master-key` to specify the old master key. The config format for `security.encryption.previous-master-key` is the same as `encryption.master-key`. On restart TiKV must access both of the old and new master key, but once TiKV is up and running, TiKV will only need access to the new key. It is okay to leave the `encryption.previous-master-key` config in the config file from that on. Even on restart, TiKV will only try to use the old key if it fail to decrypt existing data using the new master key. +To rotate master key, you have to specify both of the new master key and old master key in the config, and restart TiKV. Use `security.encryption.master-key` to specify the new master key, and use `security.encryption.previous-master-key` to specify the old master key. The config format for `security.encryption.previous-master-key` is the same as `encryption.master-key`. On restart TiKV must access both of the old and new master key, but once TiKV is up and running, TiKV will only need access to the new key. It is okay to leave the `encryption.previous-master-key` config in the config file from that on. Even on restart, TiKV only tries to use the old key if it fails to decrypt existing data using the new master key. Currently online master key rotation is not supported, so you need to restart TiKV. It is advised to do a rolling restart to a running TiKV cluster serving online query. From 4c112c98e921ab77680d9260df84480ac7c97476 Mon Sep 17 00:00:00 2001 From: yiwu-arbug Date: Mon, 18 May 2020 14:45:07 -0700 Subject: [PATCH 28/30] Update encryption-at-rest.md Co-authored-by: Ran --- encryption-at-rest.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/encryption-at-rest.md b/encryption-at-rest.md index 1fd3f5f3fc599..13a08fd2c9566 100644 --- a/encryption-at-rest.md +++ b/encryption-at-rest.md @@ -119,7 +119,7 @@ To enable S3 server-side encryption when backup to S3 using BR, pass `--s3.sse` ./br backup full --pd --storage "s3:///" --s3.sse aws:kms ``` -To use a custom AWS KMS CMK that you created and owned, pass `--s3.sse-kms-key-id` in addition. In this case, both the BR process and all the TiKV nodes in the cluster would need access to the KMS CMK (e.g. via AWS IAM), and the KMS CMK needs to be in the same AWS region as the S3 bucket used for store the backup. It is adviced to grant access to the KMS CMK to BR process and TiKV nodes via AWS IAM. Please refer to AWS documentation for usage of IAM. Example: +To use a custom AWS KMS CMK that you created and owned, pass `--s3.sse-kms-key-id` in addition. In this case, both the BR process and all the TiKV nodes in the cluster would need access to the KMS CMK (e.g. via AWS IAM), and the KMS CMK needs to be in the same AWS region as the S3 bucket used to store the backup. It is advised to grant access to the KMS CMK to BR process and TiKV nodes via AWS IAM. Refer to AWS documentation for usage of [IAM](https://docs.aws.amazon.com/IAM/latest/UserGuide/introduction.html). For example: ``` ./br backup full --pd --storage "s3:///" --s3.region --s3.sse aws:kms --s3.sse-kms-key-id 0987dcba-09fe-87dc-65ba-ab0987654321 From 063d3df88bf7a9db871ba9c4b9d1f2dcee41afb7 Mon Sep 17 00:00:00 2001 From: Yi Wu Date: Wed, 20 May 2020 02:57:55 +0800 Subject: [PATCH 29/30] remove copy button Signed-off-by: Yi Wu --- encryption-at-rest.md | 4 ---- 1 file changed, 4 deletions(-) diff --git a/encryption-at-rest.md b/encryption-at-rest.md index 13a08fd2c9566..f66ece8295f60 100644 --- a/encryption-at-rest.md +++ b/encryption-at-rest.md @@ -43,8 +43,6 @@ Regardless of data encryption method, data keys are encrypted using AES256 in GC To enable encryption, you can add the encryption section in TiKV's config file: -{{< copyable "" >}} - ``` [security.encryption] data-encryption-method = aes128-ctr @@ -55,8 +53,6 @@ Possible values for `data-encryption-method` are "aes128-ctr", "aes192-ctr", "ae The master key has to be specified if encryption is enabled (i.e. `data-encryption-method` is not "plaintext"). To specify a AWS KMS CMK as master key, add the `encryption.master-key` section after the `encryption` section: -{{< copyable "" >}} - ``` [security.encryption.master-key] type = "kms" From abdc0709653faa9a08a1a86ffa5d0f29a54b8f16 Mon Sep 17 00:00:00 2001 From: Yi Wu Date: Wed, 27 May 2020 05:45:24 +0800 Subject: [PATCH 30/30] update TOC.md Signed-off-by: Yi Wu --- TOC.md | 1 + 1 file changed, 1 insertion(+) diff --git a/TOC.md b/TOC.md index 9498a94de02ab..d90034cec58ba 100644 --- a/TOC.md +++ b/TOC.md @@ -72,6 +72,7 @@ - [Enable TLS For MySQL Clients](/encrypted-connections-with-tls-protocols.md) - [Enable TLS Between TiDB Components](/enable-tls-between-components.md) - [Generate Self-signed Certificates](/generate-self-signed-certificates.md) + - [Encryption-At-Rest](/encryption-at-rest.md) + Monitor - [Overview](/tidb-monitoring-framework.md) - [Monitor a TiDB Cluster](/monitor-a-tidb-cluster.md)