Enable leader election by default. by mattmoor · Pull Request #1476 · knative/pkg

mattmoor · 2020-07-10T20:51:35Z

This consolidates the core of sharedmain around the new leaderelection logic, which will now be enabled by default.

This can now be disabled with --disable-ha or by passing sharedmain.WithHADisabled(ctx) to sharedmain.MainWithConfig.

/assign @pmorie @vagababov @markusthoemmes

This consolidates the core of sharedmain around the new leaderelection logic, which will now be **enabled by default**. This can now be disabled with `--disable-ha` or by passing `sharedmain.WithHADisabled(ctx)` to `sharedmain.MainWithConfig`.

mattmoor · 2020-07-10T22:31:21Z

/hold

While I stage things

mattmoor · 2020-07-11T00:12:00Z

Downstream serving PR: knative/serving#8602
Downstream eventing PR: knative/eventing#3564

vagababov

/lgmt
/hold
if you want to upgrade to issue number.

vagababov · 2020-07-11T00:30:09Z

-		controllers, _ := ControllersAndWebhooksFromCtors(ctx, cmw, ctors...)
-		WatchLoggingConfigOrDie(ctx, cmw, logger, atomicLevel, component)
-		WatchObservabilityConfigOrDie(ctx, cmw, profilingHandler, logger, component)
+	// TODO(mattmoor): Remove this once HA is stable.


issue rather?

vagababov · 2020-07-11T03:02:53Z

/lgtm

yanweiguo

/lgtm

I guess the title should be Enable leader election by default?

yanweiguo · 2020-07-11T04:26:33Z

-func WebhookMainWithConfig(ctx context.Context, component string, cfg *rest.Config, ctors ...injection.ControllerConstructor) {
+// MainWithConfig runs the generic main flow for controllers and webhooks
+// with the given config.
+func MainWithConfig(ctx context.Context, component string, cfg *rest.Config, ctors ...injection.ControllerConstructor) {


If leader election mode is enabled, before this change:

webhook is running on bucket level leader election mode.

other controllers are running on pod level leader election mode.

After this change, all controllers are running on bucket level leader election mode. Is my understading right?

This race was uncovered by the chaos duck on knative/serving! When we have enabled a feature flag, e.g. multi-container, and the webhook pods are restarted, there is a brief window where the webhook is up and healthy before the configmaps have synchronized and the new webhook pod realizes the feature is enabled.

mattmoor · 2020-07-11T16:21:11Z

I pushed one additional change that was uncovered by the chaos testing downstream on serving. With this, I've had at least one clean run of serving (rerunning now) and just pushed it to the eventing PR.

vagababov

/lgtm

vaikas · 2020-07-13T16:06:47Z

/lgtm
/approve

/hold
Just a hold if you want to address the nit, up to you and remove the hold if you want to address later/never :)

knative-prow-robot · 2020-07-13T16:07:06Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mattmoor, vaikas

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [mattmoor,vaikas]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

knative-metrics-robot · 2020-07-13T16:25:20Z

The following is the coverage report on the affected files.
Say /test pull-knative-pkg-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
leaderelection/config.go	80.0%	78.1%	-1.9

vagababov · 2020-07-13T16:49:25Z

/lgtm

mattmoor · 2020-07-13T19:32:25Z

/hold cancel

I am going to land this now, and after lunch I'll start rebasing and landing the downstream PRs.

Enable HA by default.

591ba44

This consolidates the core of sharedmain around the new leaderelection logic, which will now be **enabled by default**. This can now be disabled with `--disable-ha` or by passing `sharedmain.WithHADisabled(ctx)` to `sharedmain.MainWithConfig`.

knative-prow-robot assigned markusthoemmes, pmorie and vagababov Jul 10, 2020

mattmoor commented Jul 10, 2020

View reviewed changes

knative-prow-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jul 10, 2020

knative-prow-robot requested review from markusthoemmes and n3wscott July 10, 2020 20:51

knative-prow-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 10, 2020

vagababov reviewed Jul 10, 2020

View reviewed changes

Comment thread injection/sharedmain/main.go Outdated

Comment thread injection/sharedmain/main.go Outdated

vagababov comments, build failure

c679dd5

googlebot added the cla: yes Indicates the PR's author has signed the CLA. label Jul 10, 2020

mattmoor commented Jul 10, 2020

View reviewed changes

mattmoor mentioned this pull request Jul 10, 2020

Remove the EnabledComponents field. #1478

Closed

Open an issue for enabledComponents removal.

004bb4f

knative-prow-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 10, 2020

mattmoor mentioned this pull request Jul 10, 2020

Enable HA by default knative/eventing#3564

Merged

vagababov reviewed Jul 11, 2020

View reviewed changes

mattmoor mentioned this pull request Jul 11, 2020

Add a new "chaosduck" e2e test tool for leaderelection. #1333

Merged

knative-prow-robot added the lgtm Indicates that a PR is ready to be merged. label Jul 11, 2020

yanweiguo reviewed Jul 11, 2020

View reviewed changes

knative-prow-robot assigned yanweiguo Jul 11, 2020

mattmoor changed the title ~~Enable HA by default.~~ Enable leader election by default. Jul 11, 2020

knative-prow-robot removed the lgtm Indicates that a PR is ready to be merged. label Jul 11, 2020

vagababov reviewed Jul 11, 2020

View reviewed changes

knative-prow-robot added the lgtm Indicates that a PR is ready to be merged. label Jul 11, 2020

vaikas reviewed Jul 13, 2020

View reviewed changes

Comment thread injection/sharedmain/main.go Outdated

knative-prow-robot assigned vaikas Jul 13, 2020

Drop the import alias

672516d

knative-prow-robot removed the lgtm Indicates that a PR is ready to be merged. label Jul 13, 2020

knative-prow-robot added the lgtm Indicates that a PR is ready to be merged. label Jul 13, 2020

knative-prow-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 13, 2020

knative-prow-robot merged commit a817277 into knative:master Jul 13, 2020

mattmoor deleted the single-sharedmain branch July 13, 2020 19:53

Harwayne mentioned this pull request Aug 11, 2020

Inject the config into the context. #1610

Merged

nishpa214 mentioned this pull request Sep 2, 2020

Make DefaultThreadsPerController, QPS and Burst configurable via flags tektoncd/pipeline#3156

Merged

4 tasks

Conversation

mattmoor commented Jul 10, 2020

Uh oh!

Uh oh!

Uh oh!

mattmoor commented Jul 10, 2020

Uh oh!

mattmoor commented Jul 11, 2020

Uh oh!

vagababov left a comment

Choose a reason for hiding this comment

Uh oh!

vagababov Jul 11, 2020

Choose a reason for hiding this comment

Uh oh!

vagababov commented Jul 11, 2020

Uh oh!

yanweiguo left a comment

Choose a reason for hiding this comment

Uh oh!

yanweiguo Jul 11, 2020

Choose a reason for hiding this comment

Uh oh!

mattmoor Jul 11, 2020

Choose a reason for hiding this comment

Uh oh!

mattmoor commented Jul 11, 2020

Uh oh!

vagababov left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vaikas commented Jul 13, 2020

Uh oh!

knative-prow-robot commented Jul 13, 2020

Uh oh!

knative-metrics-robot commented Jul 13, 2020

Uh oh!

vagababov commented Jul 13, 2020

Uh oh!

mattmoor commented Jul 13, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants