From 66b3da684130dfe0346a60006cbcf8eb5f311102 Mon Sep 17 00:00:00 2001 From: "Doroszlai, Attila" Date: Mon, 3 Jul 2023 20:10:23 +0200 Subject: [PATCH 1/2] HDDS-8975. Clarify SCM HA auto-bootstrap doc --- hadoop-hdds/docs/content/feature/SCM-HA.md | 33 ++++++++++++++++------ 1 file changed, 25 insertions(+), 8 deletions(-) diff --git a/hadoop-hdds/docs/content/feature/SCM-HA.md b/hadoop-hdds/docs/content/feature/SCM-HA.md index 3f509e4ca6b0..893256a24ed1 100644 --- a/hadoop-hdds/docs/content/feature/SCM-HA.md +++ b/hadoop-hdds/docs/content/feature/SCM-HA.md @@ -91,28 +91,45 @@ For reliable HA support choose 3 independent nodes to form a quorum. ## Bootstrap -The initialization of the **first** SCM-HA node is the same as a none-HA SCM: +The initialization of the **first** SCM-HA node is the same as a non-HA SCM: ``` -bin/ozone scm --init +ozone scm --init ``` Second and third nodes should be *bootstrapped* instead of init. These clusters will join to the configured RAFT quorum. The id of the current server is identified by DNS name or can be set explicitly by `ozone.scm.node.id`. Most of the time you don't need to set it as DNS based id detection can work well. ``` -bin/ozone scm --bootstrap +ozone scm --bootstrap ``` +Note: both commands perform one-time initialization. SCM still needs to be started by running `ozone scm`. + ## Auto-bootstrap -In some environment -- such as containerized / K8s environment -- we need to have a common, unified way to initialize SCM HA quorum. As a remained, the standard initialization flow is the following: +In some environments (e.g. Kubernetes) we need to have a common, unified way to initialize SCM HA quorum. As a reminder, the standard initialization flow is the following: + + 1. On the first, "primordial" node: `ozone scm --init` + 2. On second/third nodes: `ozone scm --bootstrap` + +This can be improved: primordial SCM can be configured by setting `ozone.scm.primordial.node.id` in the config to one of the nodes. + +```XML + + ozone.scm.primordial.node.id + scm1 + +``` - 1. On the first, "primordial" node, call `scm --init` - 2. On second/third nodes call `scm --bootstrap` +With this configuration both `scm --init` and `scm --bootstrap` can be safely executed on **all** SCM nodes. Each node will only perform the action applicable to it based on the `ozone.scm.primordial.node.id` and its own node ID. -This can be changed with using `ozone.scm.primordial.node.id`. You can define the primordial node. After setting this node, you should execute **both** `scm --init` and `scm --bootstrap` on **all** nodes. +Note: SCM still needs to be started by running `ozone scm` after the init/bootstrap process. -Based on the `ozone.scm.primordial.node.id`, the init process will be ignored on the second/third nodes and bootstrap process will be ignored on all nodes except the primordial one. +``` +ozone scm --init +ozone scm --bootstrap +ozone scm +``` ## SCM HA Security From 53224afdae67329a23a57dfae77bcab1b09759c3 Mon Sep 17 00:00:00 2001 From: "Doroszlai, Attila" Date: Wed, 5 Jul 2023 11:25:00 +0200 Subject: [PATCH 2/2] --daemon start --- hadoop-hdds/docs/content/feature/SCM-HA.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/hadoop-hdds/docs/content/feature/SCM-HA.md b/hadoop-hdds/docs/content/feature/SCM-HA.md index 893256a24ed1..551c340409bc 100644 --- a/hadoop-hdds/docs/content/feature/SCM-HA.md +++ b/hadoop-hdds/docs/content/feature/SCM-HA.md @@ -103,7 +103,7 @@ Second and third nodes should be *bootstrapped* instead of init. These clusters ozone scm --bootstrap ``` -Note: both commands perform one-time initialization. SCM still needs to be started by running `ozone scm`. +Note: both commands perform one-time initialization. SCM still needs to be started by running `ozone scm --daemon start`. ## Auto-bootstrap @@ -123,14 +123,16 @@ This can be improved: primordial SCM can be configured by setting `ozone.scm.pri With this configuration both `scm --init` and `scm --bootstrap` can be safely executed on **all** SCM nodes. Each node will only perform the action applicable to it based on the `ozone.scm.primordial.node.id` and its own node ID. -Note: SCM still needs to be started by running `ozone scm` after the init/bootstrap process. +Note: SCM still needs to be started after the init/bootstrap process. ``` ozone scm --init ozone scm --bootstrap -ozone scm +ozone scm --daemon start ``` +For Docker/Kubernetes, use `ozone scm` to start it in the foreground. + ## SCM HA Security ![SCM Secure HA](scm-secure-ha.png)