Skip to content

Measure container restarts over an interval instead of absolute#25115

Merged
openshift-merge-robot merged 1 commit intoopenshift:masterfrom
smarterclayton:test_over_interval
Jun 17, 2020
Merged

Measure container restarts over an interval instead of absolute#25115
openshift-merge-robot merged 1 commit intoopenshift:masterfrom
smarterclayton:test_over_interval

Conversation

@smarterclayton
Copy link
Copy Markdown
Contributor

We run these tests after disruptive events where restarts are
necessary. Instead of looking at absolute restart count, capture
the number of initial restarts and then compare subsequent runs
to ensure the count is smaller. We double the time interval and
halve the number of failing restarts in order to ensure we capture
slow restart failures (full backoff is 5m, but we can catch slow
but not limited backoff).

@openshift-ci-robot
Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: smarterclayton

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 15, 2020
@smarterclayton
Copy link
Copy Markdown
Contributor Author

/retest

@wking
Copy link
Copy Markdown
Member

wking commented Jun 15, 2020

images:

Warning: Pull failed, retrying in 5s ...
error: build error: failed to pull image: After retrying 2...501ab66378d954ee8617: unauthorized: authentication required

Dunno what that's about.

@smarterclayton
Copy link
Copy Markdown
Contributor Author

/retest

1 similar comment
@smarterclayton
Copy link
Copy Markdown
Contributor Author

/retest

@hexfusion
Copy link
Copy Markdown
Contributor

/test e2e-gcp-upgrade

@hexfusion
Copy link
Copy Markdown
Contributor

fips

Jun 16 14:53:01.193 E ns/e2e-test-build-webhooks-d2ph5 pod/pushbuild-1-build node/ip-10-0-190-41.us-west-1.compute.internal container/git-clone container exited with code 1 (Error): Cloning "http://my.docker/build" ...\nerror: fatal: unable to access 'http://my.docker/build/': Could not resolve host: my.docker; Unknown error\n

https://search.apps.build01.ci.devcluster.openshift.com/?search=fatal%3A+unable+to+access+%27http%3A%2F%2Fmy.docker&maxAge=48h&context=1&type=bug%2Bjunit&name=&maxMatches=5&maxBytes=20971520&groupBy=job

We run these tests after disruptive events where restarts are
necessary. Instead of looking at absolute restart count, capture
the number of initial restarts and then compare subsequent runs
to ensure the count is smaller. We double the time interval and
halve the number of failing restarts in order to ensure we capture
slow restart failures (full backoff is 5m, but we can catch slow
but not limited backoff).
@smarterclayton
Copy link
Copy Markdown
Contributor Author

GCP passed, going to mark this green to unblock e2e-aws-disruptive

@smarterclayton smarterclayton added the lgtm Indicates that a PR is ready to be merged. label Jun 16, 2020
@openshift-bot
Copy link
Copy Markdown
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

3 similar comments
@openshift-bot
Copy link
Copy Markdown
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Copy Markdown
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Copy Markdown
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@hexfusion
Copy link
Copy Markdown
Contributor

/retest

@openshift-bot
Copy link
Copy Markdown
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-merge-robot openshift-merge-robot merged commit 7885c46 into openshift:master Jun 17, 2020
@hexfusion
Copy link
Copy Markdown
Contributor

/cherry-pick release-4.5

@openshift-cherrypick-robot
Copy link
Copy Markdown

@hexfusion: new pull request created: #25152

Details

In response to this:

/cherry-pick release-4.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-cherrypick-robot
Copy link
Copy Markdown

@hexfusion: failed to push cherry-picked changes in GitHub: pushing failed, output: "To https://github.com/openshift-cherrypick-robot/origin\n ! [rejected] cherry-pick-25115-to-release-4.5 -> cherry-pick-25115-to-release-4.5 (fetch first)\nerror: failed to push some refs to 'https://openshift-cherrypick-robot:CENSORED@github.com/openshift-cherrypick-robot/origin'\nhint: Updates were rejected because the remote contains work that you do\nhint: not have locally. This is usually caused by another repository pushing\nhint: to the same ref. You may want to first integrate the remote changes\nhint: (e.g., 'git pull ...') before pushing again.\nhint: See the 'Note about fast-forwards' in 'git push --help' for details.\n", error: exit status 1

Details

In response to this:

/cherrypick release-4.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@hexfusion
Copy link
Copy Markdown
Contributor

/cherry-pick release-4.4

@openshift-cherrypick-robot
Copy link
Copy Markdown

@hexfusion: #25115 failed to apply on top of branch "release-4.4":

error: Failed to merge in the changes.
Using index info to reconstruct a base tree...
M	test/extended/operators/cluster.go
M	test/extended/util/annotate/generated/zz_generated.annotations.go
Falling back to patching base and 3-way merge...
Auto-merging test/extended/util/annotate/generated/zz_generated.annotations.go
CONFLICT (content): Merge conflict in test/extended/util/annotate/generated/zz_generated.annotations.go
Auto-merging test/extended/operators/cluster.go
CONFLICT (content): Merge conflict in test/extended/operators/cluster.go
Patch failed at 0001 Measure container restarts over an interval instead of absolute

Details

In response to this:

/cherry-pick release-4.4

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants