Skip to content

Bug 1731263: Re-enable preemption test with fixes#23050

Closed
ravisantoshgudimetla wants to merge 2 commits intoopenshift:masterfrom
ravisantoshgudimetla:fix-tests
Closed

Bug 1731263: Re-enable preemption test with fixes#23050
ravisantoshgudimetla wants to merge 2 commits intoopenshift:masterfrom
ravisantoshgudimetla:fix-tests

Conversation

@ravisantoshgudimetla
Copy link
Copy Markdown
Contributor

Re-enable the preemption tests with fixes to ensure they can pass.

/cc @sjenning @wking

@openshift-ci-robot openshift-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Jun 5, 2019
@ravisantoshgudimetla ravisantoshgudimetla force-pushed the fix-tests branch 2 times, most recently from 01e77c0 to e746b49 Compare June 6, 2019 16:16
milliCPU = milliCPU * 40 / 100
memAllocatable, found := currentNode.Status.Allocatable["memory"]
// Just to be tolerant use 0.6 of resources available on the node
milliCPU = int64(float64(milliCPU-currentCpuUsage) * float64(0.6))
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/cc @sjenning

}
if milliCPU <= 0 {
milliCPU = 0
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how will the low priority pod be preempted if it doesn't make any resource requests (i.e. runs best-effort)?

Copy link
Copy Markdown
Contributor Author

@ravisantoshgudimetla ravisantoshgudimetla Jun 6, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I have mentioned in the comment, we shouldn't enter this but if we're it means we're going past what can be allocated on the node.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

then why not just fail cleanly at this point rather than in some weird way that could be mistaken as a flake?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's true. The only thing I had in my mind back then was to ensure that tests won't fail because of -ve values being assigned to cpu and memory. I will remove it after the tests successfully pass.

@ravisantoshgudimetla ravisantoshgudimetla force-pushed the fix-tests branch 2 times, most recently from 8b6c3be to 4014009 Compare June 7, 2019 20:17
@openshift-ci-robot openshift-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jun 7, 2019
@ravisantoshgudimetla
Copy link
Copy Markdown
Contributor Author

/retest

@openshift-ci-robot
Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: ravisantoshgudimetla
To complete the pull request process, please assign smarterclayton
You can assign the PR to them by writing /assign @smarterclayton in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ravisantoshgudimetla ravisantoshgudimetla force-pushed the fix-tests branch 2 times, most recently from b3954cf to 689355f Compare June 9, 2019 23:59
@openshift-ci-robot
Copy link
Copy Markdown

openshift-ci-robot commented Jun 10, 2019

@ravisantoshgudimetla: The following test failed, say /retest to rerun them all:

Test name Commit Details Rerun command
ci/prow/e2e-aws-serial 7db1d5e link /test e2e-aws-serial

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

memory = int64(float64(memory-currentMemUsage) * float64(0.6))
podRes = v1.ResourceList{}
// If a node is already heavily utilized let not's create a pod there.
if milliCPU <= 0 {
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and I need to add memory check as well.

continue
}
podRes[v1.ResourceCPU] = *resource.NewMilliQuantity(int64(milliCPU), resource.DecimalSI)
podRes[v1.ResourceMemory] = *resource.NewQuantity(int64(memory), resource.BinarySI)
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test is currently failing because of this change and I will revert it. Without this we were getting the following error:

skip [k8s.io/kubernetes/test/e2e/scheduling/preemption.go:234]: We need atleast two pods to be created butall nodes are already heavily utilized, so preemption tests cannot be run

Perhaps we need to increase the node size now since we're skipping those tests?

@openshift-ci-robot openshift-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 5, 2019
@openshift-ci-robot
Copy link
Copy Markdown

@ravisantoshgudimetla: PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@stevekuznetsov
Copy link
Copy Markdown
Contributor

/retitle Bug 1731263: Re-enable preemption test with fixes

@openshift-ci-robot openshift-ci-robot changed the title Re-enable preemption test with fixes Bug 1731263: Re-enable preemption test with fixes Aug 13, 2019
@openshift-ci-robot openshift-ci-robot added the bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. label Aug 13, 2019
@openshift-ci-robot
Copy link
Copy Markdown

@ravisantoshgudimetla: This pull request references a valid Bugzilla bug. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Bug 1731263: Re-enable preemption test with fixes

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift openshift deleted a comment from openshift-ci-robot Aug 13, 2019
@soltysh
Copy link
Copy Markdown
Contributor

soltysh commented Aug 16, 2019

@ravisantoshgudimetla will you be able to rebase this PR, or @damemi feel free to pick it up

@damemi
Copy link
Copy Markdown
Contributor

damemi commented Aug 21, 2019

Rebased in #23645, this can be closed

@soltysh
Copy link
Copy Markdown
Contributor

soltysh commented Aug 22, 2019

/close

@openshift-ci-robot
Copy link
Copy Markdown

@soltysh: Closed this PR.

Details

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants