ci-operator/jobs/openshift/origin: Restore mutable properties for 4.8 and 4.9

[wking]

As discussed in e6b339e (#19513) and similar work, the current branch-forking procedure does not track these job properties which are not covered by ci-operator/config/... entries. This PR attempts to recover some of them for openshift/origin. Generated with a new script (also included in this PR):

$ hack/unify-job-properties.py ci-operator/jobs/openshift/origin
$ make jobs

I haven't gone through and guessed at the mutable properties for which sibling branches disagree, but that shouldn't block us moving ahead to repair where there is agreement.

[bparees]

it's important we assert some consistency here because when a job is optional/not-always-run in master and required in an older release, what happens is the job bitrots in master, starts permfailing, and then no one can get their backports through the older release.

so i'm a big plus one on:

1. reconciling the current state at this PR attempts to do
2. DPTP having better controls to ensure we don't fall out of sync in the future on branches.

What are we doing about #2 going forward?

we currently have a mess in 4.8 because we have a bunch of required/always-run jobs that never pass and seemingly no one noticed because they aren't always-run/required in master (yeah i know, we didn't branch master that long ago, so it may not be the whole cause).

[wking]

My impression was that #19861 and #19862 were part of the DPTP effort to avoid the need for the sort of repair I'm doing here.

[bparees]

can we get some doc (comments in the code if nothing else?) on what this does/how to invoke it?

[wking]

Added in a79d855534 -> 4e6afd259e.

[bparees]

/approve

linter isn't happy

[bparees]

how does it decide which branch to compare 4.8+4.9 to? (master vs 4.6 vs 4.7?)

[wking]

it only automatically updates if all of 4.6, 4.7, and master agree on the mutable value for that job

[wking]

otherwise it just logs the inconsistency in case you want to manually adjust

[bparees]

got it

[wking]

linter isn't happy

I've fixed two of the lint issues with 4e6afd259e -> b68c62ba84. I'm not sure if there's anything I can do about:

hack/unify-job-properties.py:7:0: E0401: Unable to import 'ruamel.yaml' (import-error)

And with that effectively unfixable, I am leaving Redefining name 'directory' from outer scope alone, instead of inventing function-specific alternative names. Checking for precedent, I don't see any comments to make the linter happy around this existing ruamel import, so I expect the linter hiccups here will not cause issues for future PRs touching other files.

[bparees]

/lgtm

[bparees]

And with that effectively unfixable, I am leaving Redefining name 'directory' from outer scope alone, instead of inventing function-specific alternative names. Checking for precedent, I don't see any comments to make the linter happy around this existing ruamel import, so I expect the linter hiccups here will not cause issues for future PRs touching other files.

the linter deliberately ignores hack/lib:

release/ci-operator/jobs/openshift/release/openshift-release-master-presubmits.yaml

Line 1064 in 590ef5b

- --ignore=lib

so this will still be an issue for subsequent PRs

[bparees]

/lgtm cancel

(still need to sort out the linter issue so that all subsequent PRs to this repo don't have a failing linter test to look at)

[bparees]

(I guess it will only be subsequent PRs that touch .py files, but even so i'd rather fix the linter job now than pile up tech debt for someone to investigate later)

[petr-muller]

/retest-required

[wking]

the linter deliberately ignores hack/lib....

Rebased onto master and shifted the script under hack/lib/ with b68c62ba84 -> 916e7c5.

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: petr-muller, wking

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [petr-muller]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

@wking: Updated the job-config-4.8 configmap in namespace ci at cluster app.ci using the following files:

• key openshift-origin-release-4.8-presubmits.yaml using file ci-operator/jobs/openshift/origin/openshift-origin-release-4.8-presubmits.yaml

Details

In response to this:

As discussed in e6b339e (#19513) and similar work, the current branch-forking procedure does not track these job properties which are not covered by ci-operator/config/... entries. This PR attempts to recover some of them for openshift/origin. Generated with a new script (also included in this PR):

$ hack/unify-job-properties.py ci-operator/jobs/openshift/origin
$ make jobs

I haven't gone through and guessed at the mutable properties for which sibling branches disagree, but that shouldn't block us moving ahead to repair where there is agreement.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.