Skip to content

Conversation

@jianlinliu
Copy link
Contributor

In https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-nightly-4.9-e2e-aws-proxy/1415490000736227328, get this error:

level=fatal msg=failed to fetch Master Machines: failed to load asset "Install Config": [controlPlane.platform.aws.zones: Invalid value: []string{"us-east-1f", "us-east-1a"}: No subnets provided for zones [us-east-1f], compute[0].platform.aws.zones: Invalid value: []string{"us-east-1f", "us-east-1a"}: No subnets provided for zones [us-east-1f]]

That is because in ./ci-operator/step-registry/ipi/conf/aws/ipi-conf-aws-commands.sh, it randomly selected "us-east-1f", "us-east-1a" as test target zones, but in the following ./ci-operator/step-registry/ipi/conf/aws/blackholenetwork/ipi-conf-aws-blackholenetwork-commands.sh, it is using vpc cf to create subnets, while the cf is creating subnets for zones in order, e.g: when AvailabilityZoneCount = 2, the cf will select the top 2 zones to create subnets, that means it is not random, that lead to no subnets created for "us-east-1f", so the error is thrown out.

This PR is fixing that issue

@openshift-ci openshift-ci bot requested review from staebler and wking July 19, 2021 05:26
@jianlinliu
Copy link
Contributor Author

cc @wking @e-tienne for review.

@jianlinliu
Copy link
Contributor Author

/retest

Copy link
Contributor

@e-tienne e-tienne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Glad you found this issue!

I think we should use the same zones across the board for all the steps.

Right not, we are using still using different zones in ipi-conf-aws and ipi-conf-aws-blackholenetwork

@jianlinliu
Copy link
Contributor Author

@wking Is there some other things need I do to merge this PR?

- name: ZONES_COUNT
default: "2"
documentation: |-
The number of AZs to present in the cluster. Accepted values are 1 and 3.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: 2 is the default, so it's not that likely that callers would set it explicitly. But it's probably worth including it in the set of acceptable values anyway, like Accepted values are 1, 2, and 3..

mapfile -t AVAILABILITY_ZONES < <(aws --region "${REGION}" ec2 describe-availability-zones | jq -r '.AvailabilityZones[] | select(.State == "available") | .ZoneName' | sort -u)
ZONES=("${AVAILABILITY_ZONES[@]:0:${ZONES_COUNT}}")
ZONES_STR="[ "
ZONES_STR+=$(join_by , "${ZONES[@]}")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit with the existing master code you're copying here: this would be simpler as:

ZONES_STR="[ $(join_by , "${ZONES[@]}") ]"

@e-tienne
Copy link
Contributor

e-tienne commented Aug 3, 2021

re-testing e2e-aws-proxy for consistency
/test e2e-aws-proxy

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Aug 3, 2021

@e-tienne: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

  • /test ci-secret-bootstrap-config-validation
  • /test app-ci-config-dry
  • /test arm01-dry
  • /test boskos-config
  • /test boskos-config-generation
  • /test build01-dry
  • /test build02-dry
  • /test ci-operator-config
  • /test ci-operator-config-metadata
  • /test ci-operator-registry
  • /test ci-secret-generator-config
  • /test ci-testgrid-allow-list
  • /test config
  • /test core-valid
  • /test correctly-sharded-config
  • /test deprecate-templates
  • /test generated-config
  • /test generated-dashboards
  • /test hive-dry
  • /test ordered-prow-config
  • /test owners
  • /test prow-config
  • /test prow-config-filenames
  • /test prow-config-semantics
  • /test release-config
  • /test release-controller-config
  • /test secret-generator-config-valid
  • /test services-valid
  • /test step-registry-metadata
  • /test step-registry-shellcheck
  • /test pylint
  • /test yamllint

The following commands are available to trigger optional jobs:

  • /test pj-rehearse
  • /test vsphere-dry

Use /test all to run the following jobs that were automatically triggered:

  • pull-ci-openshift-release-ci-secret-bootstrap-config-validation
  • pull-ci-openshift-release-master-app-ci-config-dry
  • pull-ci-openshift-release-master-arm01-dry
  • pull-ci-openshift-release-master-boskos-config
  • pull-ci-openshift-release-master-boskos-config-generation
  • pull-ci-openshift-release-master-build01-dry
  • pull-ci-openshift-release-master-build02-dry
  • pull-ci-openshift-release-master-ci-operator-config
  • pull-ci-openshift-release-master-ci-operator-config-metadata
  • pull-ci-openshift-release-master-ci-operator-registry
  • pull-ci-openshift-release-master-ci-secret-generator-config
  • pull-ci-openshift-release-master-config
  • pull-ci-openshift-release-master-core-valid
  • pull-ci-openshift-release-master-correctly-sharded-config
  • pull-ci-openshift-release-master-deprecate-templates
  • pull-ci-openshift-release-master-generated-config
  • pull-ci-openshift-release-master-generated-dashboards
  • pull-ci-openshift-release-master-hive-dry
  • pull-ci-openshift-release-master-ordered-prow-config
  • pull-ci-openshift-release-master-owners
  • pull-ci-openshift-release-master-pj-rehearse
  • pull-ci-openshift-release-master-prow-config
  • pull-ci-openshift-release-master-prow-config-filenames
  • pull-ci-openshift-release-master-prow-config-semantics
  • pull-ci-openshift-release-master-release-config
  • pull-ci-openshift-release-master-release-controller-config
  • pull-ci-openshift-release-master-secret-generator-config-valid
  • pull-ci-openshift-release-master-services-valid
  • pull-ci-openshift-release-master-step-registry-metadata
  • pull-ci-openshift-release-master-step-registry-shellcheck
  • pull-ci-openshift-release-master-vsphere-dry
  • pull-ci-openshift-release-yamllint
Details

In response to this:

re-testing e2e-aws-proxy for consistency
/test e2e-aws-proxy

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@e-tienne
Copy link
Contributor

e-tienne commented Aug 3, 2021

/test pj-rehearse

@jianlinliu
Copy link
Contributor Author

jianlinliu commented Aug 3, 2021

I polish this PR to put the blackhole step first, and have the install-config-generating step pull the zones off the pre-existing infra.

Go through the ci jobs triggered by ci robot, the failed job sound like not related to this PR change. Also https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_release/20359/rehearse-20359-pull-ci-openshift-installer-release-4.9-e2e-aws-shared-vpc/1422474381963038720 and https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_release/20359/rehearse-20359-periodic-ci-openshift-release-master-nightly-4.9-e2e-aws-proxy/1422474399247765504 proved the PR is working well.

Pls have a review.

@bparees
Copy link
Contributor

bparees commented Aug 3, 2021

/approve

@e-tienne
Copy link
Contributor

e-tienne commented Aug 3, 2021

Confirmed e2e-aws-proxy and e2e-aws-shared-vpc built the infra successfully again; will monitor future jobs for consistency.
/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Aug 3, 2021
@openshift-ci openshift-ci bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 4, 2021
fi

cat >> "${CONFIG}" << EOF
PATCH="${SHARED_DIR}/install-config-common.yaml.patch"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I think we want PATCH to live in /tmp, see #20865 where I try to fix up our existing steps. I'm fine rebasing my PR if yours lands first with the SHARED_DIR here, though.

Copy link
Contributor Author

@jianlinliu jianlinliu Aug 4, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought I can find these patch file in job's artifacts for quick troubleshooting, but sound like I did not find out them. So I am okay with moving these patch files to /tmp.

Yeah, I prefer to land this PR firstly, then your rebase them. Thanks in advance.

@jianlinliu
Copy link
Contributor Author

jianlinliu commented Aug 6, 2021

@wking I think I already resolved all the issues mentioned in review, can you help double confirm? Or some other things I missed to move forward for merging?

@stbenjam
Copy link
Member

stbenjam commented Aug 9, 2021

I see this failing a bunch on aws-proxy jobs the last few days:

 level=fatal msg= No subnets provided for zones [us-east-1d], compute[0].platform.aws.zones: Invalid value: []string{"us-east-1c", "us-east-1d"}: No subnets provided for zones [us-east-1d]] 

@stbenjam
Copy link
Member

stbenjam commented Aug 9, 2021

/retitle Bug 1991730: fixing zones mismatch issue

@openshift-ci openshift-ci bot changed the title fixing zones mismatch issue Bug 1991730: fixing zones mismatch issue Aug 9, 2021
@openshift-ci openshift-ci bot added the bugzilla/severity-high Referenced Bugzilla bug's severity is high for the branch this PR is targeting. label Aug 9, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Aug 9, 2021

@jianlinliu: This pull request references Bugzilla bug 1991730, which is invalid:

  • expected the bug to target the "4.9.0" release, but it targets "---" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

Details

In response to this:

Bug 1991730: fixing zones mismatch issue

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci bot added the bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. label Aug 9, 2021
@stbenjam
Copy link
Member

stbenjam commented Aug 9, 2021

/bugzilla refresh

@openshift-ci openshift-ci bot added the bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. label Aug 9, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Aug 9, 2021

@stbenjam: This pull request references Bugzilla bug 1991730, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.9.0) matches configured target release for branch (4.9.0)
  • bug is in the state NEW, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)
Details

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci bot removed the bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. label Aug 9, 2021
@jstuever
Copy link
Contributor

jstuever commented Aug 9, 2021

/assign

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Aug 17, 2021

@jianlinliu: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Rerun command
ci/rehearse/periodic-ci-openshift-release-master-nightly-4.9-e2e-aws-proxy 6534446 link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-nightly-4.9-e2e-aws-ovn-local-gateway fb4a42f link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-nightly-4.9-e2e-aws-single-node 863c751 link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-ci-4.9-upgrade-from-stable-4.8-e2e-aws-ovn-upgrade 863c751 link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-nightly-4.9-e2e-aws-csi a796ad7 link /test pj-rehearse
ci/rehearse/openshift/kubernetes/master/k8s-e2e-aws-serial a796ad7 link /test pj-rehearse
ci/rehearse/openshift/kubernetes/master/k8s-e2e-aws a796ad7 link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-ci-4.9-e2e-aws-calico a796ad7 link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-ci-4.9-e2e-aws-upgrade-single-node a796ad7 link /test pj-rehearse
ci/rehearse/openshift/origin/release-4.1/e2e-aws-image-ecosystem 550f6a3 link /test pj-rehearse
ci/rehearse/openshift/origin/release-4.1/e2e-aws-builds 550f6a3 link /test pj-rehearse
ci/rehearse/openshift/installer/release-4.9/e2e-aws-upgrade 550f6a3 link /test pj-rehearse
ci/rehearse/openshift/sdn/release-4.9/e2e-aws-multitenant 550f6a3 link /test pj-rehearse
ci/rehearse/redhat-developer/jenkins-operator/main/e2e 550f6a3 link /test pj-rehearse
ci/rehearse/red-hat-storage/odf-operator/main/odf-operator-bundle-e2e-aws 550f6a3 link /test pj-rehearse
ci/rehearse/openshift/origin/release-4.9/e2e-aws 550f6a3 link /test pj-rehearse
ci/rehearse/operator-framework/operator-marketplace/release-4.9/e2e-aws-upgrade 550f6a3 link /test pj-rehearse
ci/rehearse/operator-framework/operator-marketplace/release-4.9/e2e-aws-serial 550f6a3 link /test pj-rehearse
ci/rehearse/openshift/machine-config-operator/release-4.9/e2e-aws-techpreview-featuregate 550f6a3 link /test pj-rehearse
ci/rehearse/openshift/cloud-credential-operator/release-4.9/e2e-aws-manual-oidc 550f6a3 link /test pj-rehearse
ci/rehearse/openshift/origin/release-4.2/e2e-cmd 550f6a3 link /test pj-rehearse
ci/rehearse/openshift/cluster-network-operator/release-4.9/e2e-ovn-ipsec-step-registry 550f6a3 link /test pj-rehearse
ci/rehearse/openshift/builder/release-4.9/e2e-aws-cgroupsv2 550f6a3 link /test pj-rehearse
ci/rehearse/openshift/aws-efs-csi-driver-operator/release-4.9/operator-e2e 550f6a3 link /test pj-rehearse
ci/rehearse/openshift/origin/release-4.9/e2e-aws-jenkins 550f6a3 link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-nightly-4.9-e2e-aws-workers-rhel7 550f6a3 link /test pj-rehearse
ci/rehearse/openshift/kubernetes/release-4.9/configmap-scale 550f6a3 link /test pj-rehearse
ci/rehearse/openshift/ovn-kubernetes/release-4.9/e2e-ovn-hybrid-step-registry 550f6a3 link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-nightly-4.9-e2e-aws-csi-migration 550f6a3 link /test pj-rehearse
ci/rehearse/openshift/cluster-cloud-controller-manager-operator/release-4.9/e2e-aws-ccm-install 550f6a3 link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-stable-4.8-upgrade-from-stable-4.6-e2e-aws-upgrade-paused 550f6a3 link /test pj-rehearse
ci/rehearse/openshift/cluster-logging-operator/tech-preview/e2e-operator 550f6a3 link /test pj-rehearse
ci/rehearse/openshift/origin/release-4.9/e2e-aws-disruptive 550f6a3 link /test pj-rehearse
ci/rehearse/openshift/windows-machine-config-operator/release-4.9/aws-e2e-upgrade 550f6a3 link /test pj-rehearse
ci/prow/pj-rehearse 550f6a3 link /test pj-rehearse

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Copy link
Contributor

@vrutkovs vrutkovs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Aug 20, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Aug 20, 2021

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bparees, e-tienne, jianlinliu, vrutkovs

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 20, 2021
@vrutkovs
Copy link
Contributor

/hold

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 20, 2021
@jianlinliu
Copy link
Contributor Author

close this PR, use #21145 to replace this one.

@jianlinliu jianlinliu closed this Aug 20, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Aug 20, 2021

@jianlinliu: This pull request references Bugzilla bug 1991730. The bug has been updated to no longer refer to the pull request using the external bug tracker.

Details

In response to this:

Bug 1991730: fixing zones mismatch issue

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. bugzilla/severity-high Referenced Bugzilla bug's severity is high for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants