e2e-vsphere: use vm template built from ova by staebler · Pull Request #3591 · openshift/release

staebler · 2019-04-25T02:58:09Z

Instead of using a pre-existing VM template in the vSphere cluster, create a new VM template using the OVA referenced in rhcos.json. This will use the boot image appropriate for the installer being tested.

staebler · 2019-04-25T02:58:53Z

/hold

This requires openshift/installer#1673 in order for the rhcos.json file to be present in the UPI installer image.

staebler · 2019-04-25T02:59:26Z

/cc @dav1x @vrutkovs

vrutkovs · 2019-04-25T14:20:58Z

/test pj-rehearse

dav1x · 2019-04-25T14:42:08Z

/lgtm

Instead of using a pre-existing VM template in the vSphere cluster, create a new VM template using the OVA referenced in rhcos.json. This will use the boot image appropriate for the installer being tested.

vrutkovs

/lgtm

openshift-ci-robot · 2019-04-26T13:55:21Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dav1x, staebler, vrutkovs

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~ci-operator/templates/openshift/installer/OWNERS~~ [staebler]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

vrutkovs · 2019-04-26T13:55:27Z

/test pj-rehearse

vrutkovs · 2019-04-29T09:05:36Z

/test pj-rehearse

droslean · 2019-04-29T13:33:14Z

/test pj-rehearse

staebler · 2019-04-30T18:03:10Z

/test pj-rehearse

vrutkovs · 2019-05-02T09:24:31Z

Failing tests:

[Feature:Platform] Managed cluster should have no crashlooping pods in core namespaces over two minutes [Suite:openshift/conformance/parallel]
[cli] oc adm must-gather runs successfully [Suite:openshift/conformance/parallel]

/test pj-rehearse

vrutkovs · 2019-05-02T13:10:22Z

fail [github.com/openshift/origin/test/extended/operators/cluster.go:109]: Expected
    <[]string | len:1, cap:1>: [
        "Pod openshift-sdn/sdn-b9m65 is not healthy: conflict, name \"k8s_sdn_sdn-b9m65_openshift-sdn_710463b4-6b74-11e9-825e-0050569b3174_2\" already reserved for ctr \"d2a062adae0dfd31505ac8a9c54be7b9a82266ef8180f5bfa6771c29ac10758f\"",
    ]
to be empty

seems to flake often on vsphere tests, cc @openshift/sig-networking

vrutkovs · 2019-05-02T13:10:36Z

/test pj-rehearse

vrutkovs · 2019-05-02T14:25:27Z

vsphere test passes
/hold cancel

openshift-ci-robot · 2019-05-02T14:32:42Z

@staebler: Updated the following 2 configmaps:

prow-job-cluster-launch-installer-upi-e2e configmap in namespace ci using the following files:
- key cluster-launch-installer-upi-e2e.yaml using file ci-operator/templates/openshift/installer/cluster-launch-installer-upi-e2e.yaml
prow-job-cluster-launch-installer-upi-e2e configmap in namespace ci-stg using the following files:
- key cluster-launch-installer-upi-e2e.yaml using file ci-operator/templates/openshift/installer/cluster-launch-installer-upi-e2e.yaml

Details

In response to this:

Instead of using a pre-existing VM template in the vSphere cluster, create a new VM template using the OVA referenced in rhcos.json. This will use the boot image appropriate for the installer being tested.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

danwinship · 2019-05-02T14:48:22Z

"Pod openshift-sdn/sdn-b9m65 is not healthy: conflict, name "k8s_sdn_sdn-b9m65_openshift-sdn_710463b4-6b74-11e9-825e-0050569b3174_2" already reserved for ctr "d2a062adae0dfd31505ac8a9c54be7b9a82266ef8180f5bfa6771c29ac10758f"",

That error appears to come from cri-o. The problem might not be sdn-specific, it's just that it's coming from a worker node, and sdn is one of the first things that gets deployed to the workers after they're brought up (since most other pods either (a) only schedule to masters, or (b) require pod networking to be ready before they run).

The only noticeably odd thing I see in either the sdn or node logs is the fact that the nodes are being brought up with an incorrect system time, which then gets fixed when chrony starts:

Apr 30 18:13:33 compute-0 chronyd[901]: Selected source 91.148.192.49
Apr 30 18:13:33 compute-0 chronyd[901]: System clock TAI offset set to 37 seconds
Apr 30 18:13:33 compute-0 chronyd[901]: System clock wrong by 106.526472 seconds, adjustment started
Apr 30 18:15:19 compute-0 chronyd[901]: System clock was stepped by 106.526472 seconds

(In particular, this means that compute-0 shuts down at 18:21:11 and comes back up at 18:19:30. It's possible that something is getting upset about this.)

squeed · 2019-05-02T15:52:17Z

The timecube strikes again.

openshift-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Apr 25, 2019

openshift-ci-robot requested review from abhinavdahiya and crawford April 25, 2019 02:58

openshift-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 25, 2019

openshift-ci-robot requested review from dav1x and vrutkovs April 25, 2019 02:59

dav1x approved these changes Apr 25, 2019

View reviewed changes

openshift-ci-robot assigned dav1x Apr 25, 2019

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Apr 25, 2019

vrutkovs reviewed Apr 25, 2019

View reviewed changes

Comment thread ci-operator/templates/openshift/installer/cluster-launch-installer-upi-e2e.yaml Outdated

staebler force-pushed the e2e_vsphere-use_correct_rhcos branch from 9c7f882 to ae38706 Compare April 25, 2019 15:20

openshift-ci-robot removed the lgtm Indicates that a PR is ready to be merged. label Apr 25, 2019

vrutkovs reviewed Apr 25, 2019

View reviewed changes

Comment thread ci-operator/templates/openshift/installer/cluster-launch-installer-upi-e2e.yaml Outdated

vrutkovs reviewed Apr 25, 2019

View reviewed changes

Comment thread ci-operator/templates/openshift/installer/cluster-launch-installer-upi-e2e.yaml Outdated

staebler force-pushed the e2e_vsphere-use_correct_rhcos branch 2 times, most recently from 2a3b513 to b0868d3 Compare April 25, 2019 17:43

vrutkovs reviewed Apr 25, 2019

View reviewed changes

Comment thread ci-operator/templates/openshift/installer/cluster-launch-installer-upi-e2e.yaml Outdated

staebler force-pushed the e2e_vsphere-use_correct_rhcos branch from b0868d3 to 5dd024f Compare April 26, 2019 13:14

e2e-vsphere: use vm template built from ova

77743c1

Instead of using a pre-existing VM template in the vSphere cluster, create a new VM template using the OVA referenced in rhcos.json. This will use the boot image appropriate for the installer being tested.

staebler force-pushed the e2e_vsphere-use_correct_rhcos branch from 5dd024f to 77743c1 Compare April 26, 2019 13:16

openshift-ci-robot assigned vrutkovs Apr 26, 2019

vrutkovs approved these changes Apr 26, 2019

View reviewed changes

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Apr 26, 2019

openshift-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 2, 2019

openshift-merge-robot merged commit 332f83f into openshift:master May 2, 2019

Conversation

staebler commented Apr 25, 2019

Uh oh!

staebler commented Apr 25, 2019

Uh oh!

staebler commented Apr 25, 2019

Uh oh!

vrutkovs commented Apr 25, 2019

Uh oh!

dav1x commented Apr 25, 2019

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vrutkovs left a comment

Choose a reason for hiding this comment

Uh oh!

openshift-ci-robot commented Apr 26, 2019

Uh oh!

vrutkovs commented Apr 26, 2019

Uh oh!

vrutkovs commented Apr 29, 2019

Uh oh!

droslean commented Apr 29, 2019

Uh oh!

staebler commented Apr 30, 2019

Uh oh!

vrutkovs commented May 2, 2019

Uh oh!

vrutkovs commented May 2, 2019

Uh oh!

vrutkovs commented May 2, 2019

Uh oh!

vrutkovs commented May 2, 2019

Uh oh!

openshift-ci-robot commented May 2, 2019

Uh oh!

danwinship commented May 2, 2019

Uh oh!

squeed commented May 2, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants