transient risk analysis

[dgoodwin]

Adds a new path through the risk_analysis API where we do not need to have the prow job run already imported into sippy. This will allow us to run a risk analysis at the end of a prow job run, and store it's result as an artifact.

The previous path of getting analysis by ID will soon be adjusted to return the risk analysis at the point in time we imported the job. This will come in a future PR.

Caller needs make a GET request with a json request body containing something like:

{
    "ProwJob": {
        "Name": "periodic-ci-openshift-release-master-nightly-4.12-console-aws"
    },
    "URL": "",
    "Tests": [
        {
            "Test": {
                "Name": "[sig-arch] Managed cluster should set requests but not limits [Suite:openshift/conformance/parallel]"
            },
            "Suite": {
                "Name": "openshift-tests-upgrade"
            },
            "Status": 12
        },
        {
            "Test": {
                "Name": "[sig-arch] events should not repeat pathologically"
            },
            "Suite": {
                "Name": "openshift-tests-upgrade"
            },
            "Status": 12
        }
    ]
}

These files will soon be written out by openshift-tests in a PR that I will link shortly. A new openshift-tests risk-analysis command will also be added to handle the semantics of merging the above json files (there are often two), and making the HTTP request to sippy, then storing the resulting risk-analysis.json as an artifact.

For posterity, risk analysis looks as follows:

{
  "ProwJobName": "periodic-ci-openshift-release-master-nightly-4.12-console-aws",
  "ProwJobRunID": 0,
  "Tests": [
    {
      "Name": "[sig-arch] Managed cluster should set requests but not limits [Suite:openshift/conformance/parallel]",
      "Risk": {
        "Level": {
          "Name": "Unknown",
          "Level": 7
        },
        "Reasons": [
          "Unable to find matching test results for variants: [amd64 aws ha ovn]"
        ]
      },
      "OpenBugs": [
        {
          "id": 14863787,
          "key": "OCPBUGS-1801",
          "created_at": "2022-09-28T14:47:31.323454-03:00",
          "updated_at": "2022-10-23T20:40:23.681878-03:00",
          "deleted_at": null,
          "status": "ON_QA",
          "last_change_time": "2022-10-13T04:38:22-03:00",
          "summary": "ibm-powervs-block-csi-driver-controller does not set resource requests",
          "affects_versions": [
            "4.12.0"
          ],
          "fix_versions": [],
          "components": [
            "Storage"
          ],
          "url": "https://issues.redhat.com/browse/OCPBUGS-1801"
        }
      ]
    },
    {
      "Name": "[sig-arch] events should not repeat pathologically",
      "Risk": {
        "Level": {
          "Name": "Unknown",
          "Level": 7
        },
        "Reasons": [
          "Unable to find matching test results for variants: [amd64 aws ha ovn]"
        ]
      },
      "OpenBugs": [
        {
          "id": 14814726,
          "key": "OCPBUGS-587",
          "created_at": "2022-10-18T11:30:25.302407-03:00",
          "updated_at": "2022-10-23T20:40:22.303217-03:00",
          "deleted_at": null,
          "status": "ASSIGNED",
          "last_change_time": "2022-10-21T11:17:23-03:00",
          "summary": "FailedToUpdateEndpointSlices Error updating Endpoint Slices for Service ",
          "affects_versions": [
            "4.12"
          ],
          "fix_versions": [],
          "components": [
            "kube-apiserver"
          ],
          "url": "https://issues.redhat.com/browse/OCPBUGS-587"
        }
      ]
    },
    {
      "Name": "[sig-arch][bz-storage][Late] Alerts alert/KubePersistentVolumeErrors should not be at or above pending [Suite:openshift/conformance/parallel]",
      "Risk": {
        "Level": {
          "Name": "Medium",
          "Level": 5
        },
        "Reasons": [
          "This test has passed 93.94% of 33 runs on release 4.12 [amd64 aws ha ovn] in the last week."
        ]
      },
      "OpenBugs": []
    }
  ],
  "OverallRisk": {
    "Level": {
      "Name": "Unknown",
      "Level": 7
    },
    "Reasons": [
      "Maximum failed test risk: Unknown"
    ]
  },
  "OpenBugs": []
}

TRT-603

[dgoodwin]

/assign @neisw

Forrest could you take a pass at this sometime this week and let me know what you think?

[neisw]

/lgtm

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dgoodwin, neisw

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [dgoodwin]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

@dgoodwin: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.