Skip to content

Bug 1731263: Disable failing preemption e2es#24868

Merged
openshift-merge-robot merged 1 commit intoopenshift:masterfrom
damemi:disable-preemption-e2es
Apr 21, 2020
Merged

Bug 1731263: Disable failing preemption e2es#24868
openshift-merge-robot merged 1 commit intoopenshift:masterfrom
damemi:disable-preemption-e2es

Conversation

@damemi
Copy link
Copy Markdown
Contributor

@damemi damemi commented Apr 13, 2020

Main BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1731263. BZ contains discussion on failure causes, usually that the "preemptor" pod does not land on the expected "victim" pod's node, and so it does not evict the pod we expect it to.

These tests have been disabled for several releases, and were recently re-enabled by 5f52d7e#diff-6ba77494282f6e840f44b01ce97335afL88

After being reenabled, they are still failing: https://testgrid.k8s.io/redhat-openshift-ocp-release-4.5-informing#release-openshift-ocp-installer-e2e-azure-serial-4.5

These have had the following attempted fixes:

There was an upstream issue for some of these tests here kubernetes/kubernetes#88441. They do not currently flake in upstream CI runs

There are some comments in this PR explaining how the test currently assigns a pod to a node with affinity: kubernetes/kubernetes#90118. I believe my attempt to fix this by manually setting the NodeName won't actually work because that bypasses scheduling, and so preemption does not run.

@damemi
Copy link
Copy Markdown
Contributor Author

damemi commented Apr 13, 2020

/cc @soltysh @smarterclayton

Copy link
Copy Markdown
Contributor

@soltysh soltysh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not a fan of disabling test but it looks like there's a major problem with this test upstream that should be fixed first than we can rely on the actual results. The band-aid that upstream did to prevent flakes doesn't apparently solve the root cause.
/lgtm
/approve
/hold
you'll need to fix the verify script

@openshift-ci-robot openshift-ci-robot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Apr 14, 2020
@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Apr 14, 2020
@damemi
Copy link
Copy Markdown
Contributor Author

damemi commented Apr 14, 2020

@soltysh what are you referring to? there doesn't appear to be any major problems with the test upstream, and I think the issue I linked was actually caused by us trying to patch up the test to work for our clusters

@damemi damemi force-pushed the disable-preemption-e2es branch from 3532feb to f8c378c Compare April 14, 2020 12:46
@openshift-ci-robot openshift-ci-robot removed the lgtm Indicates that a PR is ready to be merged. label Apr 14, 2020
@damemi
Copy link
Copy Markdown
Contributor Author

damemi commented Apr 14, 2020

/retest

2 similar comments
@damemi
Copy link
Copy Markdown
Contributor Author

damemi commented Apr 14, 2020

/retest

@damemi
Copy link
Copy Markdown
Contributor Author

damemi commented Apr 15, 2020

/retest

@soltysh
Copy link
Copy Markdown
Contributor

soltysh commented Apr 21, 2020

/retitle Bug 1731263: Disable failing preemption e2es

Copy link
Copy Markdown
Contributor

@soltysh soltysh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/retest

@soltysh
Copy link
Copy Markdown
Contributor

soltysh commented Apr 21, 2020

/hold cancel

@openshift-ci-robot openshift-ci-robot added lgtm Indicates that a PR is ready to be merged. and removed do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. labels Apr 21, 2020
@openshift-ci-robot
Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: damemi, soltysh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot openshift-ci-robot changed the title Disable failing preemption e2es Bug 1731263: Disable failing preemption e2es Apr 21, 2020
@openshift-ci-robot openshift-ci-robot added the bugzilla/severity-low Referenced Bugzilla bug's severity is low for the branch this PR is targeting. label Apr 21, 2020
@openshift-ci-robot
Copy link
Copy Markdown

@damemi: This pull request references Bugzilla bug 1731263, which is valid. The bug has been updated to refer to the pull request using the external bug tracker.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.5.0) matches configured target release for branch (4.5.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)
Details

In response to this:

Bug 1731263: Disable failing preemption e2es

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci-robot openshift-ci-robot added the bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. label Apr 21, 2020
@openshift-bot
Copy link
Copy Markdown
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

1 similar comment
@openshift-bot
Copy link
Copy Markdown
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-merge-robot openshift-merge-robot merged commit 595206b into openshift:master Apr 21, 2020
@openshift-ci-robot
Copy link
Copy Markdown

@damemi: Some pull requests linked via external trackers have merged: . The following pull requests linked via external trackers have not merged:

Details

In response to this:

Bug 1731263: Disable failing preemption e2es

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. bugzilla/severity-low Referenced Bugzilla bug's severity is low for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants