coordinator: distributed deployment with auto-recovery#1373
Merged
Conversation
katexochen
added
enterprise feature
and removed
feature
burgerdev
force-pushed
the
burgerdev/ha
branch
2 times, most recently
from
c2dd477 to
8e18fb3
Compare
burgerdev
force-pushed
the
burgerdev/ha
branch
2 times, most recently
from
f8f7351 to
caff06d
Compare
|
katexochen
reviewed
May 20, 2025
Comment on lines
+29
to
+44
| config, err := rest.InClusterConfig() | ||
| if err != nil { | ||
| return err | ||
| } | ||
| clientset, err := kubernetes.NewForConfig(config) | ||
| if err != nil { | ||
| return err | ||
| } | ||
| namespace, err := os.ReadFile("/var/run/secrets/kubernetes.io/serviceaccount/namespace") | ||
| if err != nil { | ||
| return err | ||
| } | ||
| discovery := peerdiscovery.New(clientset, string(namespace)) | ||
| return peerrecovery.New(c.guard, discovery, c.issuer, c.httpsGetter, c.logger).RunRecovery(ctx) |
Member
Author
There was a problem hiding this comment.
I added a logging statement about the recovery routine starting, hope this is what you meant.
Contributor
There was a problem hiding this comment.
Meant start and error logging, the error logging is still missing.
burgerdev
force-pushed
the
burgerdev/ha
branch
3 times, most recently
from
8a497f8 to
59040aa
Compare
burgerdev
changed the title
burgerdev
commented
May 21, 2025
|
|
||
| ## Workflow | ||
|
|
||
| <!-- TODO(burgerdev): this how-to anticipates planned changes to readiness which are not yet committed. --> |
Contributor
There was a problem hiding this comment.
Nice, as that is already approved we can delete the comment here.
katexochen
added
feature
katexochen
changed the title
katexochen
approved these changes
May 21, 2025
|
|
||
| ```sh | ||
| kubectl rollout restart statefulset/coordinator | ||
| sleep 2m |
Contributor
There was a problem hiding this comment.
nit: can we wait for the rollout to finish here instead of sleeping?
| ``` | ||
|
|
||
| This will create additional Coordinator instances, one after another, as described in the [Kubernetes documentation for `StatefulSet`](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#deployment-and-scaling-guarantees). | ||
| After some time, the additional Coordinators should enter the `Ready` state, too. |
Contributor
There was a problem hiding this comment.
nit: could we add a command to wait for readiness?
|
|
||
| ## Workflow | ||
|
|
||
| <!-- TODO(burgerdev): this how-to anticipates planned changes to readiness which are not yet committed. --> |
Contributor
There was a problem hiding this comment.
Nice, as that is already approved we can delete the comment here.
Co-authored-by: Paul Meyer <katexochen0@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds support for deploying multiple replicas of the Coordinator.
Further reading:
Testing:
just e2e openssl AKS-CLH-SNP