raft: add support for using backend for ha_storage by calvn · Pull Request #9193 · hashicorp/vault

calvn · 2020-06-10T22:03:23Z

No description provided.

…rage

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>

…rage # Conflicts: # vault/logical_system.go

alexanderbez · 2020-06-19T20:11:03Z

+			// If we encountered and error we should try to create the key
+			// again.
+			if backoff {
+				nextRotationTime = time.Now().Add(10 * time.Second)


Just curious, is the 10 seconds here arbitrary? How many times do we foresee backoff to be hit before success?

Yeah, this is just an arbitrary value that's short enough for a retry. I'm assuming this is to guard against any potential errors encountered during rotation, such as inability to access physical storage due to some transient failure. @briankassouf might have more context on this.

Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>

alexanderbez

Performed another review. Looks good @calvn 👍

Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>

briankassouf · 2020-06-23T01:10:58Z

-		c.logger.Error("failed to create raft TLS keyring", "error", err)
-		return nil, err
+	if raftBackend != nil {
+		if _, err := c.raftCreateTLSKeyring(ctx); err != nil {


You've wrapped these functions in that nil check a few places, but don't they already check if we are running raft storage in the function?

Yeah, it's not necessary to wrap in this case. The other call is based on onInit though, so I'll leave that untouched.

Actually I've been trying to get away from returning (nil, nil) as a pattern. I'll keep this as-is and return (nil, error) instead on that method.

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

vishalnayak

This is looking great!

briankassouf

🎉

…rage

* master: Entity and alias counts (#9262) Token gauge metrics implementation. (#9239) mfa: fix import path on test file (#9303) doc: update vault helm enterprise image examples (#9299) raft: add support for using backend for ha_storage (#9193) Document new and previously undocumented telemetry metrics: (#9283) Improve the performance of snapshot installs by using rename (#9247) docs: add additional info around transform for tweak and template type (#9203) Update CHANGELOG.md CL++: Add go version to server message output

* raft: initial work on raft ha storage support * add note on join * add todo note * raft: add support for bootstrapping and joining existing nodes * raft: gate bootstrap join by reading leader api address from storage * raft: properly check for raft-only for certain conditionals * raft: add bootstrap to api and cli * raft: fix bootstrap cli command * raft: add test for setting up new cluster with raft HA * raft: extend TestRaft_HA_NewCluster to include inmem and consul backends * raft: add test for updating an existing cluster to use raft HA * raft: remove debug log lines, clean up verifyRaftPeers * raft: minor cleanup * raft: minor cleanup * Update physical/raft/raft.go Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com> * Update vault/ha.go Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com> * Update vault/ha.go Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com> * Update vault/logical_system_raft.go Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com> * Update vault/raft.go Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com> * Update vault/raft.go Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com> * address feedback comments * address feedback comments * raft: refactor tls keyring logic * address feedback comments * Update vault/raft.go Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com> * Update vault/raft.go Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com> * address feedback comments * testing: fix import ordering * raft: rename var, cleanup comment line * docs: remove ha_storage restriction note on raft * docs: more raft HA interaction updates with migration and recovery mode * docs: update the raft join command * raft: update comments * raft: add missing isRaftHAOnly check for clearing out state set earlier * raft: update a few ha_storage config checks * Update command/operator_raft_bootstrap.go Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com> * raft: address feedback comments * raft: fix panic when checking for config.HAStorage.Type * Update vault/raft.go Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com> * Update website/pages/docs/commands/operator/raft.mdx Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com> * raft: remove bootstrap cli command * Update vault/raft.go Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com> * Update vault/raft.go Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com> * raft: address review feedback * raft: revert vendored sdk * raft: don't send applied index and node ID info if we're HA-only Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com> Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com> Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>

calvn added 13 commits May 26, 2020 16:46

raft: initial work on raft ha storage support

a01d02d

add note on join

1c0d821

add todo note

a4fce98

raft: add support for bootstrapping and joining existing nodes

17edc2c

raft: gate bootstrap join by reading leader api address from storage

5cf1915

raft: properly check for raft-only for certain conditionals

d2e9857

Merge remote-tracking branch 'origin/master' into feature-raft-ha-sto…

29a27e5

…rage

raft: add bootstrap to api and cli

cb3c8e0

raft: fix bootstrap cli command

c80d6ad

raft: add test for setting up new cluster with raft HA

83beaf4

raft: extend TestRaft_HA_NewCluster to include inmem and consul backends

187af61

raft: add test for updating an existing cluster to use raft HA

6a2fb4b

raft: remove debug log lines, clean up verifyRaftPeers

155e67f

calvn added this to the 1.5 milestone Jun 10, 2020

calvn requested review from briankassouf, mjarmy and vishalnayak June 10, 2020 22:03

calvn added 2 commits June 10, 2020 16:17

Merge remote-tracking branch 'origin/master' into feature-raft-ha-sto…

19622c3

…rage

raft: minor cleanup

317ce9d

briankassouf reviewed Jun 12, 2020

View reviewed changes

calvn and others added 8 commits June 12, 2020 17:09

raft: minor cleanup

dcd97ed

Update physical/raft/raft.go

282e53c

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

Update vault/ha.go

be61b48

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

Update vault/ha.go

568d9fc

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

Update vault/logical_system_raft.go

8bb5d20

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

Update vault/raft.go

fe55165

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

Update vault/raft.go

415ae01

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

address feedback comments

b30369d

alexanderbez reviewed Jun 15, 2020

View reviewed changes

Comment thread command/operator_raft_join.go Outdated

alexanderbez reviewed Jun 15, 2020

View reviewed changes

Comment thread command/server.go Outdated

calvn added 4 commits June 17, 2020 16:41

docs: update the raft join command

23606e9

raft: update comments

beb706e

raft: add missing isRaftHAOnly check for clearing out state set earlier

0f20555

raft: update a few ha_storage config checks

47d72e5

vishalnayak reviewed Jun 18, 2020

View reviewed changes

Comment thread command/operator_raft_bootstrap.go Outdated

Comment thread command/server.go Outdated

Comment thread physical/raft/raft.go Outdated

Comment thread physical/raft/raft.go

Comment thread vault/ha.go Outdated

Comment thread vault/raft.go Outdated

calvn and others added 4 commits June 18, 2020 16:13

Update command/operator_raft_bootstrap.go

5c0e909

Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>

raft: address feedback comments

26321a8

raft: fix panic when checking for config.HAStorage.Type

41618b9

Merge remote-tracking branch 'origin/master' into feature-raft-ha-sto…

00ae776

…rage # Conflicts: # vault/logical_system.go

alexanderbez reviewed Jun 19, 2020

View reviewed changes

Comment thread vault/raft.go Outdated

alexanderbez reviewed Jun 19, 2020

View reviewed changes

Update vault/raft.go

1d78660

Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>

alexanderbez reviewed Jun 19, 2020

View reviewed changes

Comment thread website/pages/docs/commands/operator/raft.mdx Outdated

alexanderbez reviewed Jun 19, 2020

View reviewed changes

calvn and others added 2 commits June 19, 2020 13:31

Update website/pages/docs/commands/operator/raft.mdx

bbab0a6

Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>

raft: remove bootstrap cli command

278f9ea

briankassouf reviewed Jun 23, 2020

View reviewed changes

calvn and others added 4 commits June 22, 2020 22:46

Update vault/raft.go

0278c20

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

Update vault/raft.go

64cf79b

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

raft: address review feedback

08458ea

raft: revert vendored sdk

07ce510

vishalnayak approved these changes Jun 23, 2020

View reviewed changes

raft: don't send applied index and node ID info if we're HA-only

fd4ed94

briankassouf approved these changes Jun 23, 2020

View reviewed changes

Merge remote-tracking branch 'origin/master' into feature-raft-ha-sto…

363cad1

…rage

calvn merged commit 045836d into master Jun 23, 2020

briankassouf deleted the feature-raft-ha-storage branch June 23, 2020 21:38

ncabatoff mentioned this pull request Jul 21, 2021

vault crashes on unseal when raft is used as ha_storage #8206

Closed

Conversation

calvn commented Jun 10, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alexanderbez Jun 19, 2020

Choose a reason for hiding this comment

Uh oh!

calvn Jun 19, 2020

Choose a reason for hiding this comment

Uh oh!

Uh oh!

alexanderbez left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

briankassouf Jun 23, 2020

Choose a reason for hiding this comment

Uh oh!

calvn Jun 23, 2020

Choose a reason for hiding this comment

Uh oh!

calvn Jun 23, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vishalnayak left a comment

Choose a reason for hiding this comment

Uh oh!

briankassouf left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

calvn Jun 23, 2020 •

edited

Loading