Skip to content

[4.9] Generate ci-operator configuration#15794

Merged
openshift-merge-robot merged 5 commits intoopenshift:masterfrom
AlexNPavel:4.9-generate-configs-and-jobs
Feb 11, 2021
Merged

[4.9] Generate ci-operator configuration#15794
openshift-merge-robot merged 5 commits intoopenshift:masterfrom
AlexNPavel:4.9-generate-configs-and-jobs

Conversation

@AlexNPavel
Copy link
Copy Markdown
Contributor

@AlexNPavel AlexNPavel commented Feb 11, 2021

Create 4.9 ci-operator configuration and jobs. Using script from #12621, modified to copy from 4.8 to 4.9.

#!/bin/bash

set -o nounset
set -o errexit
set -o pipefail

git fetch origin
git reset --hard origin/master

config-brancher --config-dir ci-operator/config --current-release=4.8 --future-release=4.9 --confirm
git add ci-operator/config
git commit -m "[4.9] Generate ci-operator configuration" -m '```console
$ config-brancher --config-dir ci-operator/config --current-release=4.8 --future-release=4.9 --confirm
```'

ci-operator-config-mirror --config-path ci-operator/config --to-org openshift-priv
git add ci-operator/config
git commit -m "[4.9] Generate ci-operator configuration for openshift-priv" -m '```console
$ ci-operator-config-mirror --config-path ci-operator/config --to-org openshift-priv
```'

cat <<EOF >/tmp/filter.py
#!/usr/bin/env python3

import yaml
import sys
import copy

with open(sys.argv[1]) as f:
    all = yaml.full_load(f)
    for t in ("presubmits", "postsubmits"):
        for repo in all.get(t, {}):
            pruned = []
            for job in all.get(t, {}).get(repo, []):
                if job.get("agent", "") == "kubernetes":
                    pruned.append(job)
            all[t][repo] = pruned

with open(sys.argv[1], 'w') as f:
    yaml.dump(all, f, default_flow_style=False)
EOF

chmod +x /tmp/filter.py

find ci-operator/jobs/ -name '*-release-4.8-*submits.yaml' -or -name '*-release-4.8-periodics.yaml' | while read -r item; do
    echo -n "Processing: ${item}"
    cp "${item}" "${item/4.8/4.9}"
    sed -i 's/-release-4.8-/-release-4.9-/g' "${item/4.8/4.9}" # job names
    sed -i 's/- release-4.8/- release-4.9/g' "${item/4.8/4.9}" # branches
    /tmp/filter.py "${item/4.8/4.9}"
    echo " [DONE]"
done

# openshift/release are special and are handled separately
rm -f ci-operator/jobs/openshift/release/openshift-release-release-4.9-periodics.yaml
git add ci-operator/jobs
git commit -m "[4.9] Prepopulate jobs by copying 4.8 ones" -m '```console
$ cat <<EOF >/tmp/filter.py
#!/usr/bin/env python3

import yaml
import sys
import copy

with open(sys.argv[1]) as f:
    all = yaml.full_load(f)
    for t in ("presubmits", "postsubmits"):
        for repo in all.get(t, {}):
            pruned = []
            for job in all.get(t, {}).get(repo, []):
                if job.get("agent", "") == "kubernetes":
                    pruned.append(job)
            all[t][repo] = pruned

with open(sys.argv[1], '\''w'\'') as f:
    yaml.dump(all, f, default_flow_style=False)
EOF

$ chmod +x /tmp/filter.py

$ find ci-operator/jobs/ -name '\''*-release-4.8-*submits.yaml'\'' -or -name '\''*-release-4.8-periodics.yaml'\'' | while read -r item; do
    cp "${item}" "${item/4.8/4.9}"
    sed -i '\''s/-release-4.8-/-release-4.9-/g'\'' "${item/4.8/4.9}" # job names
    sed -i '\''s/- release-4.8/- release-4.9/g'\'' "${item/4.8/4.9}" # branches
    /tmp/filter.py "${item/4.8/4.9}"
    echo " [DONE]"
$ done
```
'

make jobs
git add ci-operator/jobs
git commit -m "[4.9] Fixup prepopulated jobs with make jobs" -m '```console
$ make jobs
```
'

make template-allowlist
git add core-services/template-deprecation/_allowlist.yaml
git commit -m "[4.9] Update template allowlist" -m '```console
$ make template-allowlist
```
'

/hold

@openshift-ci-robot openshift-ci-robot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Feb 11, 2021
```console
$ config-brancher --config-dir ci-operator/config --current-release=4.8 --future-release=4.9 --confirm
```
```console
$ ci-operator-config-mirror --config-path ci-operator/config --to-org openshift-priv
```
```console
$ cat <<EOF >/tmp/filter.py
#!/usr/bin/env python3

import yaml
import sys
import copy

with open(sys.argv[1]) as f:
    all = yaml.full_load(f)
    for t in ("presubmits", "postsubmits"):
        for repo in all.get(t, {}):
            pruned = []
            for job in all.get(t, {}).get(repo, []):
                if job.get("agent", "") == "kubernetes":
                    pruned.append(job)
            all[t][repo] = pruned

with open(sys.argv[1], 'w') as f:
    yaml.dump(all, f, default_flow_style=False)
EOF

$ chmod +x /tmp/filter.py

$ find ci-operator/jobs/ -name '*-release-4.8-*submits.yaml' -or -name '*-release-4.8-periodics.yaml' | while read -r item; do
    cp "${item}" "${item/4.8/4.9}"
    sed -i 's/-release-4.8-/-release-4.9-/g' "${item/4.8/4.9}" # job names
    sed -i 's/- release-4.8/- release-4.9/g' "${item/4.8/4.9}" # branches
    /tmp/filter.py "${item/4.8/4.9}"
    echo " [DONE]"
$ done
```
```console
$ make template-allowlist
```
@AlexNPavel AlexNPavel force-pushed the 4.9-generate-configs-and-jobs branch from b670286 to 9cc04f0 Compare February 11, 2021 18:30
@AlexNPavel
Copy link
Copy Markdown
Contributor Author

/retest

@stevekuznetsov
Copy link
Copy Markdown
Contributor

/lgtm
/hold

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Feb 11, 2021
@openshift-ci-robot
Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: AlexNPavel, stevekuznetsov

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [AlexNPavel,stevekuznetsov]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@AlexNPavel
Copy link
Copy Markdown
Contributor Author

/hold cancel

@openshift-ci-robot openshift-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 11, 2021
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Feb 11, 2021

@AlexNPavel: The following tests failed, say /retest to rerun all failed tests:

Test name Commit Details Rerun command
ci/rehearse/openshift/openshift-apiserver/release-4.9/images b67028647833c4968dc2f1102797dd0ad67998fa link /test pj-rehearse
ci/rehearse/openshift/cluster-nfd-operator/release-4.9/verify b67028647833c4968dc2f1102797dd0ad67998fa link /test pj-rehearse
ci/rehearse/openshift/insights-operator/release-4.9/e2e b67028647833c4968dc2f1102797dd0ad67998fa link /test pj-rehearse
ci/rehearse/openshift/special-resource-operator/release-4.9/unit b67028647833c4968dc2f1102797dd0ad67998fa link /test pj-rehearse
ci/rehearse/openshift/openshift-controller-manager/release-4.9/e2e-aws-proxy b67028647833c4968dc2f1102797dd0ad67998fa link /test pj-rehearse
ci/rehearse/openshift/library-go/release-4.9/verify b67028647833c4968dc2f1102797dd0ad67998fa link /test pj-rehearse
ci/rehearse/openshift/csi-external-attacher/release-4.9/e2e-aws-csi b67028647833c4968dc2f1102797dd0ad67998fa link /test pj-rehearse
ci/rehearse/periodic-ci-kubevirt-hyperconverged-cluster-operator-release-4.9-hco-e2e-deploy-nightly-master-aws b67028647833c4968dc2f1102797dd0ad67998fa link /test pj-rehearse
ci/rehearse/openshift/cluster-config-operator/release-4.9/verify-deps b67028647833c4968dc2f1102797dd0ad67998fa link /test pj-rehearse
ci/rehearse/openshift/installer/release-4.9/e2e-aws-upi b67028647833c4968dc2f1102797dd0ad67998fa link /test pj-rehearse
ci/rehearse/openshift/cluster-storage-operator/release-4.9/verify b67028647833c4968dc2f1102797dd0ad67998fa link /test pj-rehearse
ci/rehearse/openshift/builder/release-4.9/e2e-aws-builds b67028647833c4968dc2f1102797dd0ad67998fa link /test pj-rehearse
ci/rehearse/openshift/file-integrity-operator/release-4.9/images b67028647833c4968dc2f1102797dd0ad67998fa link /test pj-rehearse
ci/rehearse/openshift/cluster-baremetal-operator/release-4.9/e2e-metal-ipi b67028647833c4968dc2f1102797dd0ad67998fa link /test pj-rehearse
ci/rehearse/openshift/kubernetes/release-4.9/verify-commits b67028647833c4968dc2f1102797dd0ad67998fa link /test pj-rehearse
ci/rehearse/openshift/origin/release-4.9/e2e-gcp-csi b67028647833c4968dc2f1102797dd0ad67998fa link /test pj-rehearse
ci/rehearse/openshift/jenkins/release-4.9/images b67028647833c4968dc2f1102797dd0ad67998fa link /test pj-rehearse
ci/rehearse/openshift/compliance-operator/release-4.9/images b67028647833c4968dc2f1102797dd0ad67998fa link /test pj-rehearse
ci/rehearse/openshift/origin/release-4.9/e2e-gcp b67028647833c4968dc2f1102797dd0ad67998fa link /test pj-rehearse
ci/rehearse/openshift/cluster-monitoring-operator/release-4.9/images b67028647833c4968dc2f1102797dd0ad67998fa link /test pj-rehearse
ci/rehearse/openshift/multus-admission-controller/release-4.9/e2e-aws b67028647833c4968dc2f1102797dd0ad67998fa link /test pj-rehearse
ci/rehearse/openshift/cluster-etcd-operator/release-4.9/e2e-operator b67028647833c4968dc2f1102797dd0ad67998fa link /test pj-rehearse
ci/rehearse/openshift/project-request-limit/release-4.9/unit b67028647833c4968dc2f1102797dd0ad67998fa link /test pj-rehearse
ci/rehearse/openshift/openshift-controller-manager/release-4.9/unit b67028647833c4968dc2f1102797dd0ad67998fa link /test pj-rehearse
ci/rehearse/kube-reporting/helm/release-4.9/images b67028647833c4968dc2f1102797dd0ad67998fa link /test pj-rehearse
ci/rehearse/openshift/node-feature-discovery/release-4.9/verify b67028647833c4968dc2f1102797dd0ad67998fa link /test pj-rehearse
ci/rehearse/openshift/baremetal-runtimecfg/release-4.9/gofmt b67028647833c4968dc2f1102797dd0ad67998fa link /test pj-rehearse
ci/rehearse/openshift/router/release-4.9/e2e-agnostic b67028647833c4968dc2f1102797dd0ad67998fa link /test pj-rehearse
ci/rehearse/openshift/oc/release-4.9/unit 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/cluster-policy-controller/release-4.9/e2e-aws 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/template-service-broker/release-4.9/verify-deps 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/sriov-cni/release-4.9/e2e-aws 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/gcp-pd-csi-driver-operator/release-4.9/verify 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/cluster-api-provider-aws/release-4.9/e2e-aws-operator 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/prometheus-operator/release-4.9/test-unit 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/openshift-tests/release-4.9/images 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/ocp-release-operator-sdk/release-4.9/unit 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/ironic-rhcos-downloader/release-4.9/images 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/sriov-network-operator/release-4.9/pkg 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/template-service-broker/release-4.9/e2e-aws 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/csi-external-snapshotter/release-4.9/e2e-ovirt 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/openshift-state-metrics/release-4.9/unit 9cc04f0 link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-openshift-tests-private-release-4.9-sanity 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/csi-node-driver-registrar/release-4.9/unit 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/aws-ebs-csi-driver-operator/release-4.9/unit 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/csi-external-snapshotter/release-4.9/images 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/baremetal-runtimecfg/release-4.9/unit 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/console/release-4.9/images 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/mdns-publisher/release-4.9/e2e-metal-ipi 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/cluster-kube-apiserver-operator/release-4.9/verify-deps 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/openshift-apiserver/release-4.9/unit 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/network-metrics-daemon/release-4.9/unit 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/jenkins/release-4.9/e2e-aws 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/machine-api-operator/release-4.9/govet 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/origin/release-4.9/images 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/aws-ebs-csi-driver/release-4.9/verify 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/csi-external-attacher/release-4.9/images 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/machine-config-operator/release-4.9/e2e-aws 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/builder/release-4.9/e2e-aws 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/kubernetes/release-4.9/e2e-gcp-upgrade 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/sdn/release-4.9/e2e-aws-upgrade 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/cluster-storage-operator/release-4.9/e2e-aws-upgrade 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/coredns/release-4.9/e2e-aws-upgrade 9cc04f0 link /test pj-rehearse
ci/prow/pj-rehearse 9cc04f0 link /test pj-rehearse
ci/rehearse/openshift/coredns/release-4.9/unit 9cc04f0 link /test pj-rehearse

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@openshift-merge-robot openshift-merge-robot merged commit b13c832 into openshift:master Feb 11, 2021
@AlexNPavel AlexNPavel deleted the 4.9-generate-configs-and-jobs branch February 11, 2021 23:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants