Skip to content

Fix upgrade deadlock in Pipeline and Triggers #592

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
tekton-robot merged 1 commit into from
Jan 28, 2022
Merged

Fix upgrade deadlock in Pipeline and Triggers #592

tekton-robot merged 1 commit into from
Jan 28, 2022

Conversation

@nikhil-thomas
Copy link
Member

@nikhil-thomas nikhil-thomas commented Jan 27, 2022 •
edited
Loading

Add a mechanism to disable webhook validation if the configMaps
are not already there in the cluster.

What this patch does: when webhook endpoint slice is [], the "rules" section of config.webhook.** validatingwebhookconfiguration is remove. So that the updates on configMaps eg: config-defaults are not deadlocked.

The dead lock is caused due to a cluster level cyclic dependency in
tektoncd/pipelines and in tektoncd/triggers. More details will be captured in
issues in the both repository.

Note: infact this deadlock can occur in any tektoncd project which follows tektoncd/pipeline
file list structure in config/.

Signed-off-by: Nikhil Thomas nikthoma@redhat.com

Changes

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you
review them:

See the contribution guide for more details.

Release Notes

NONE

@tekton-robot tekton-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jan 27, 2022
@tekton-robot
Copy link
Contributor

tekton-robot commented Jan 27, 2022

The following is the coverage report on the affected files.
Say /test pull-tekton-operator-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/common/deadlockbreaker.go Do not exist 0.0%

@nikhil-thomas
Copy link
Member Author

nikhil-thomas commented Jan 27, 2022

/hold

@tekton-robot tekton-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 27, 2022
@nikhil-thomas nikhil-thomas force-pushed the fix/pipeline-webhook-deadlock branch from 0bbe19e to 05aa7f8 Compare January 27, 2022 11:53
@tekton-robot
Copy link
Contributor

tekton-robot commented Jan 27, 2022

The following is the coverage report on the affected files.
Say /test pull-tekton-operator-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/common/deadlockbreaker.go Do not exist 0.0%

@nikhil-thomas nikhil-thomas force-pushed the fix/pipeline-webhook-deadlock branch from 05aa7f8 to f2bd245 Compare January 27, 2022 11:57
@nikhil-thomas
Copy link
Member Author

nikhil-thomas commented Jan 27, 2022

/unhold

@tekton-robot
Copy link
Contributor

tekton-robot commented Jan 27, 2022

The following is the coverage report on the affected files.
Say /test pull-tekton-operator-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/common/deadlockbreaker.go Do not exist 0.0%

@nikhil-thomas
Copy link
Member Author

nikhil-thomas commented Jan 27, 2022

/hold cancel

@tekton-robot tekton-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 27, 2022
@nikhil-thomas nikhil-thomas force-pushed the fix/pipeline-webhook-deadlock branch from f2bd245 to 765f04e Compare January 27, 2022 12:06
@tekton-robot
Copy link
Contributor

tekton-robot commented Jan 27, 2022

The following is the coverage report on the affected files.
Say /test pull-tekton-operator-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/common/deadlockbreaker.go Do not exist 0.0%

@sm43
Copy link
Member

sm43 commented Jan 27, 2022

/approve
/meow

@tekton-robot
Copy link
Contributor

tekton-robot commented Jan 27, 2022

@sm43: cat image

Details

In response to this:

/approve
/meow

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@tekton-robot tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 27, 2022
sm43
sm43 reviewed Jan 27, 2022
View reviewed changes
@nikhil-thomas nikhil-thomas force-pushed the fix/pipeline-webhook-deadlock branch from 765f04e to c9195a9 Compare January 27, 2022 12:59
@tekton-robot tekton-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jan 27, 2022
@nikhil-thomas
Fix upgrade deadlock in Pipeline and Triggers
234fb43 
Add a mechanism to disable webhook validation if the configMaps
are not already there in the cluster.

What this patch does: when webhook endpoint slice is [], the "rules"
section of config.webhook.** validatingwebhookconfiguration is remove.
So that the updates on configMaps eg: config-defaults are not
deadlocked.

The dead lock is caused due to a cluster level cyclic dependency in
tektoncd/pipelines and in tektoncd/triggers. More details will be captured in
issues in the both repository.

Note: infact this deadlock can occur in any tektoncd project which follows tektoncd/pipeline
file list structure in `config/`.

Signed-off-by: Nikhil Thomas <nikthoma@redhat.com>
@nikhil-thomas nikhil-thomas force-pushed the fix/pipeline-webhook-deadlock branch from c9195a9 to 234fb43 Compare January 27, 2022 13:00
@tekton-robot
Copy link
Contributor

tekton-robot commented Jan 27, 2022

The following is the coverage report on the affected files.
Say /test pull-tekton-operator-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/common/deadlockbreaker.go Do not exist 0.0%

@tekton-robot
Copy link
Contributor

tekton-robot commented Jan 27, 2022

The following is the coverage report on the affected files.
Say /test pull-tekton-operator-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/common/deadlockbreaker.go Do not exist 0.0%

vdemeester
vdemeester approved these changes Jan 27, 2022
View reviewed changes
Copy link
Member

@vdemeester vdemeester left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM 👍🏼

pradeepitm12
pradeepitm12 approved these changes Jan 28, 2022
View reviewed changes
Copy link
Contributor

@pradeepitm12 pradeepitm12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@tekton-robot tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Jan 28, 2022
@tekton-robot
Copy link
Contributor

tekton-robot commented Jan 28, 2022

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: pradeepitm12, sm43, vdemeester

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@tekton-robot tekton-robot merged commit b0c6763 into tektoncd:main Jan 28, 2022
@nikhil-thomas nikhil-thomas mentioned this pull request Feb 2, 2022
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vdemeester vdemeester vdemeester approved these changes

@concaf concaf Awaiting requested review from concaf

+2 more reviewers

@sm43 sm43 sm43 left review comments

@pradeepitm12 pradeepitm12 pradeepitm12 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@pradeepitm12 pradeepitm12

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@nikhil-thomas @tekton-robot @sm43 @vdemeester @pradeepitm12