Skip to content

Add SNO upgrade workflows#17519

Merged
openshift-merge-robot merged 4 commits intoopenshift:masterfrom
vrutkovs:bip-upgrade
Apr 19, 2021
Merged

Add SNO upgrade workflows#17519
openshift-merge-robot merged 4 commits intoopenshift:masterfrom
vrutkovs:bip-upgrade

Conversation

@vrutkovs
Copy link
Copy Markdown
Contributor

@vrutkovs vrutkovs commented Apr 7, 2021

This setups SNO cluster on AWS and Azure from release:initial and upgrades it
to release:latest. Disruption tests are skipped, as they are not
expected to pass in SNO configuration.

Split out from #16290 to speed up review.

TODO:

@openshift-ci-robot openshift-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 7, 2021
@vrutkovs vrutkovs force-pushed the bip-upgrade branch 2 times, most recently from 00a9e85 to e08062d Compare April 7, 2021 17:02
@vrutkovs
Copy link
Copy Markdown
Contributor Author

vrutkovs commented Apr 7, 2021

Reference: https://issues.redhat.com/browse/OTA-379

@vrutkovs
Copy link
Copy Markdown
Contributor Author

vrutkovs commented Apr 7, 2021

/retest

1 similar comment
@vrutkovs
Copy link
Copy Markdown
Contributor Author

vrutkovs commented Apr 8, 2021

/retest

@vrutkovs vrutkovs force-pushed the bip-upgrade branch 4 times, most recently from a02a353 to 65334bd Compare April 8, 2021 18:28
Comment thread ci-operator/step-registry/single-node/conf/single-node-conf-commands.sh Outdated
@vrutkovs vrutkovs force-pushed the bip-upgrade branch 13 times, most recently from 18374fe to 3b75f9e Compare April 13, 2021 09:56
@vrutkovs vrutkovs changed the title WIP Add SNO upgrade workflow Add SNO upgrade workflow Apr 13, 2021
@vrutkovs vrutkovs force-pushed the bip-upgrade branch 2 times, most recently from d8c53d8 to bdf5630 Compare April 16, 2021 12:27
@vrutkovs vrutkovs changed the title WIP Add SNO upgrade workflow Add SNO upgrade workflows Apr 16, 2021
@openshift-ci-robot openshift-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 16, 2021
@vrutkovs
Copy link
Copy Markdown
Contributor Author

/hold

I'll create a new PR to fix openshift-e2e-test typo

@openshift-ci-robot openshift-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 16, 2021
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we called in single-node everywhere (rather than sno)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this in purpose?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixes a typo in previous PR - #17823

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is loki is something specific here? Just so i understand, or this is the standard flow

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copied from regular upgrade workflow

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This step would setup promtail on the cluster, sending container and journald logs to the central repo, which can be queried by Grafana.

This is a standard practice for upgrade workflows, as intermediate containers can be lost during reboot (or the node may never come back)

@romfreiman
Copy link
Copy Markdown

Lets add those jobs here as well: https://testgrid.k8s.io/redhat-single-node

omertuc and others added 4 commits April 19, 2021 17:38
This setups SNO cluster on AWS from `release:initial` and upgrades it
to `release:latest`. Disruption tests are skipped, as they are not
expected to pass in SNO configuration.

As a test this adds periodic job to be testing 4.8 CI images every 24 hours
This test suite won't run disruption tests
set -o nounset
set -o pipefail

SINGLE_NODE_AZURE_INSTANCE_TYPE="Standard_D32s_v3"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I expect there to be one set of folks maintaining these single-node conf step(s), so it might we worth a single single-node/conf command instead of the platform-specific sub-steps. On the other hand, I'm not in the maintainer set, so doesn't matter for me ;).

Copy link
Copy Markdown
Member

@wking wking left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty good to me. Remaining wrinkles can be ironed out in follow-up work.

/lgtm
/retest

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Apr 19, 2021
@hongkailiu
Copy link
Copy Markdown
Member

/approve

@openshift-ci-robot
Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hongkailiu, omertuc, vrutkovs, wking

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 19, 2021
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Apr 19, 2021

@vrutkovs: The following tests failed, say /retest to rerun all failed tests:

Test name Commit Details Rerun command
ci/rehearse/openshift/origin/master/e2e-aws-upgrade-sno 05016b4176c79f3123d2d5457423280c487f8cad link /test pj-rehearse
ci/rehearse/openshift/origin/master/e2e-azure-upgrade-sno 05016b4176c79f3123d2d5457423280c487f8cad link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-ci-4.8-e2e-aws-upgrade-sno bdf5630702c875e2edd9dbab25dc2e308e66224d link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-ci-4.8-e2e-azure-upgrade-sno bdf5630702c875e2edd9dbab25dc2e308e66224d link /test pj-rehearse
ci/rehearse/openshift/cluster-network-operator/release-4.9/e2e-azure-ovn-dualstack cc8e8d50859e731b2d2df5857785699b07ba4fcd link /test pj-rehearse
ci/rehearse/openshift/cluster-network-operator/release-4.9/e2e-ovn-ipsec-step-registry cc8e8d50859e731b2d2df5857785699b07ba4fcd link /test pj-rehearse
ci/rehearse/openshift/machine-config-operator/release-4.9/e2e-gcp-single-node cc8e8d50859e731b2d2df5857785699b07ba4fcd link /test pj-rehearse
ci/rehearse/openshift/origin/release-4.9/e2e-agnostic-cmd cc8e8d50859e731b2d2df5857785699b07ba4fcd link /test pj-rehearse
ci/rehearse/openshift/origin/release-4.9/e2e-gcp-disruptive cc8e8d50859e731b2d2df5857785699b07ba4fcd link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-ci-4.6-upgrade-from-stable-4.5-e2e-ovirt-upgrade 8827e3284d3c4220c7f3399146ae92fcc9bb1e72 link /test pj-rehearse
ci/rehearse/openshift/origin/release-4.1/e2e-aws-builds 8827e3284d3c4220c7f3399146ae92fcc9bb1e72 link /test pj-rehearse
ci/rehearse/openshift/origin/release-4.1/e2e-aws-image-ecosystem 8827e3284d3c4220c7f3399146ae92fcc9bb1e72 link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-ci-4.8-upgrade-from-stable-4.7-e2e-vsphere-upgrade 8827e3284d3c4220c7f3399146ae92fcc9bb1e72 link /test pj-rehearse
ci/rehearse/openshift/ovn-kubernetes/release-4.9/e2e-ovn-hybrid-step-registry 8827e3284d3c4220c7f3399146ae92fcc9bb1e72 link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-cluster-api-provider-kubevirt-release-4.9-sanity-ovn 8827e3284d3c4220c7f3399146ae92fcc9bb1e72 link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-nightly-4.8-e2e-aws-csi 8827e3284d3c4220c7f3399146ae92fcc9bb1e72 link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-nightly-4.8-e2e-azure-csi 8827e3284d3c4220c7f3399146ae92fcc9bb1e72 link /test pj-rehearse
ci/rehearse/openshift/origin/release-4.9/e2e-aws-jenkins 8827e3284d3c4220c7f3399146ae92fcc9bb1e72 link /test pj-rehearse
ci/rehearse/openshift/origin/release-4.2/e2e-cmd 8827e3284d3c4220c7f3399146ae92fcc9bb1e72 link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-nightly-4.8-e2e-aws-single-node 8827e3284d3c4220c7f3399146ae92fcc9bb1e72 link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-ci-4.8-upgrade-from-stable-4.7-e2e-openstack-upgrade 8827e3284d3c4220c7f3399146ae92fcc9bb1e72 link /test pj-rehearse
ci/rehearse/openshift/origin/release-4.9/e2e-aws 8827e3284d3c4220c7f3399146ae92fcc9bb1e72 link /test pj-rehearse
ci/rehearse/openshift/ovn-kubernetes/release-4.9/e2e-aws-ovn 8827e3284d3c4220c7f3399146ae92fcc9bb1e72 link /test pj-rehearse
ci/rehearse/openshift/vmware-vsphere-csi-driver/release-4.9/e2e-vsphere-csi 8827e3284d3c4220c7f3399146ae92fcc9bb1e72 link /test pj-rehearse
ci/rehearse/openshift/ovn-kubernetes/release-4.9/e2e-azure-ovn 8827e3284d3c4220c7f3399146ae92fcc9bb1e72 link /test pj-rehearse
ci/rehearse/openshift/ovn-kubernetes/release-4.9/e2e-gcp-ovn 8827e3284d3c4220c7f3399146ae92fcc9bb1e72 link /test pj-rehearse
ci/rehearse/openshift/origin/release-4.9/e2e-aws-disruptive 8827e3284d3c4220c7f3399146ae92fcc9bb1e72 link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-nightly-4.8-e2e-aws-proxy 8827e3284d3c4220c7f3399146ae92fcc9bb1e72 link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-ci-4.8-e2e-azure-cilium 8827e3284d3c4220c7f3399146ae92fcc9bb1e72 link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-ci-4.8-e2e-azure-upgrade 8827e3284d3c4220c7f3399146ae92fcc9bb1e72 link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-ci-4.8-e2e-azure-serial 8827e3284d3c4220c7f3399146ae92fcc9bb1e72 link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-ci-4.8-upgrade-from-stable-4.7-e2e-azure-ovn-upgrade 8827e3284d3c4220c7f3399146ae92fcc9bb1e72 link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-ci-4.8-upgrade-from-stable-4.7-e2e-aws-ovn-upgrade 8827e3284d3c4220c7f3399146ae92fcc9bb1e72 link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-ci-4.8-e2e-gcp-serial 8827e3284d3c4220c7f3399146ae92fcc9bb1e72 link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-ci-4.8-upgrade-from-stable-4.7-e2e-gcp-ovn-upgrade 8827e3284d3c4220c7f3399146ae92fcc9bb1e72 link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-ci-4.8-upgrade-from-stable-4.7-e2e-ovirt-upgrade 8827e3284d3c4220c7f3399146ae92fcc9bb1e72 link /test pj-rehearse
ci/rehearse/openshift/origin/release-4.9/e2e-gcp-builds 8827e3284d3c4220c7f3399146ae92fcc9bb1e72 link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-ci-4.8-e2e-azure-upgrade-single-node ad04803 link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-ci-4.8-e2e-aws-upgrade-single-node ad04803 link /test pj-rehearse
ci/prow/pj-rehearse ad04803 link /test pj-rehearse

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@vrutkovs
Copy link
Copy Markdown
Contributor Author

/hold cancel

All issues were resolved

@openshift-ci-robot openshift-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 19, 2021
@openshift-merge-robot openshift-merge-robot merged commit 857c946 into openshift:master Apr 19, 2021
@openshift-ci-robot
Copy link
Copy Markdown
Contributor

@vrutkovs: Updated the following 3 configmaps:

  • job-config-master configmap in namespace ci at cluster app.ci using the following files:
    • key openshift-release-master-periodics.yaml using file ci-operator/jobs/openshift/release/openshift-release-master-periodics.yaml
  • step-registry configmap in namespace ci at cluster app.ci using the following files:
    • key OWNERS using file ci-operator/step-registry/openshift/upgrade/aws/single-node/OWNERS
    • key openshift-upgrade-aws-single-node-workflow.metadata.json using file ci-operator/step-registry/openshift/upgrade/aws/single-node/openshift-upgrade-aws-single-node-workflow.metadata.json
    • key openshift-upgrade-aws-single-node-workflow.yaml using file ci-operator/step-registry/openshift/upgrade/aws/single-node/openshift-upgrade-aws-single-node-workflow.yaml
    • key OWNERS using file ci-operator/step-registry/openshift/upgrade/azure/single-node/OWNERS
    • key openshift-upgrade-azure-single-node-workflow.metadata.json using file ci-operator/step-registry/openshift/upgrade/azure/single-node/openshift-upgrade-azure-single-node-workflow.metadata.json
    • key openshift-upgrade-azure-single-node-workflow.yaml using file ci-operator/step-registry/openshift/upgrade/azure/single-node/openshift-upgrade-azure-single-node-workflow.yaml
    • key OWNERS using file ci-operator/step-registry/single-node/conf/azure/OWNERS
    • key single-node-conf-azure-commands.sh using file ci-operator/step-registry/single-node/conf/azure/single-node-conf-azure-commands.sh
    • key single-node-conf-azure-ref.metadata.json using file ci-operator/step-registry/single-node/conf/azure/single-node-conf-azure-ref.metadata.json
    • key single-node-conf-azure-ref.yaml using file ci-operator/step-registry/single-node/conf/azure/single-node-conf-azure-ref.yaml
  • ci-operator-master-configs configmap in namespace ci at cluster app.ci using the following files:
    • key openshift-release-master__ci-4.8.yaml using file ci-operator/config/openshift/release/openshift-release-master__ci-4.8.yaml
Details

In response to this:

This setups SNO cluster on AWS and Azure from release:initial and upgrades it
to release:latest. Disruption tests are skipped, as they are not
expected to pass in SNO configuration.

Split out from #16290 to speed up review.

TODO:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants