COS-2781: CephCapDropPanic on 4.12.54+, 4.13.36+, 4.14.14+, 4.15.0+

[petr-muller]

Clusters that use CephFS volumes are at risk until a fixed kernel is provided. Set up risks for edges between unaffected and affected versions. Regexes are a bit tricky, here are links to regex101 I used for testing:

4.12: https://regex101.com/r/wqaLUv/1
4.13: https://regex101.com/r/LpY9h1/1
4.14: https://regex101.com/r/jIfQ2g/1
4.15: https://regex101.com/r/XfSOPa/1

I have created a file for the lowest of each minor version and copied the rest like this (fish):

for r4 in (seq 15 24)
    cp blocked-edges/4.14.14-RHELKernelCephFSCapDropPanic.yaml blocked-edges/4.14.$r4-RHELKernelCephFSCapDropPanic.yaml
    sed -i "s|to: 4\.14\.14|to: 4\.14\.$r4|" blocked-edges/4.14.$r4-RHELKernelCephFSCapDropPanic.yaml
end

[openshift-ci-robot]

@petr-muller: This pull request references COS-2781 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the spike to target the "4.16.0" version, but no target version was set.

Details

In response to this:

Clusters that use CephFS volumes are at risk until a fixed kernel is provided. Set up risks for edges between unaffected and affected versions. Regexes are a bit tricky, here are links to regex101 I used for testing:

4.12: https://regex101.com/r/wqaLUv/1
4.13: https://regex101.com/r/LpY9h1/1
4.14: https://regex101.com/r/jIfQ2g/1
4.15: https://regex101.com/r/XfSOPa/1

I have created a file for the lowest of each minor version and copied the rest like this (fish):

for r4 in (seq 15 24)
   cp blocked-edges/4.14.14-RHELKernelCephFSCapDropPanic.yaml blocked-edges/4.14.$r4-RHELKernelCephFSCapDropPanic.yaml
   sed -i "s|to: 4\.14\.14|to: 4\.14\.$r4|" blocked-edges/4.14.$r4-RHELKernelCephFSCapDropPanic.yaml
end

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[petr-muller]

I only tested the PromQL on clusters without Ceph, dont have one using Ceph at hand

[wking]

nit: our only Ceph declaration so far is:

$ git grep -oh 'name: .*Ceph.*' | sort | uniq -c
     13 name: CephParallelFsync

We didn't call out RHELKernel or FS then. If we follow that pattern here, it would be CephCapDropPanic. But 🤷, no need to be particularly consistent vs. old risk names, if you prefer the additional length/context.

[petr-muller]

I followed RHELKernelHighLoadIOWait but I kinda like shorter names and maybe we dont want to give RHEL bad publicity

[wking]

nit: no need for the quotes, although they're fine if you want to keep them. You're also missing a trailing period, which we usually include in our messages.

$ for X in blocked-edges/*.yaml; do yaml2json < "${X}" | jq '(.message // "-")[-1:]'; done | sort | uniq -c | sort -n
      1 "z"
      2 "O"
      4 "e"
     12 "n"
    117 "-"
    652 "."

[petr-muller]

That's what I get by copying RHELKernelHighLoadIOWait files

[wking]

nit: can you add #3591's _id="" for HyperShift-compatibility to cluster_version and ceph_health_status?

[wking]

/lgtm

[sdodson]

/retest-required

[sdodson]

/lgtm

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: petr-muller, sdodson, wking

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [petr-muller,wking]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[petr-muller]

I think we'll need to drop 4.15.4 too

cincinnati-graph-data/channels/candidate-4.15.yaml

Lines 58 to 63 in 261733a

	- 4.15.2
	- 4.15.3
	- 4.15.5
	- 4.15.6
	- 4.15.7
	- 4.15.8

[openshift-ci]

New changes are detected. LGTM label has been removed.

[openshift-ci]

@petr-muller: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.