Add ClusterProfile field for ClusterVersion

[guillaumerose]

ClusterProfile is the identifier that the Cluster Version
Operator uses to determine which manifests to apply. Operators can
be excluded completely or can have different manifests for each supported
profile.

The following annotation is used to include manifests
for a given profile: include.release.openshift.io/[identifier]=true

If ClusterProfile is empty, self-managed-high-availability profile is used.

---

In order to use the cluster profile in the installer, we need to pass the information somehow.
A way of doing it is with a ConfigMap (ugly?) or a new field in the ClusterVersion (seems more natural).

CRC will use it as follow:

$ openshift-install create manifests
$ patch the ClusterVersion manifest
$ openshift-install create cluster

ClusterProfile could also be a required field, but it might be an issue for updating a 4.6 cluster.

[openshift-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: guillaumerose
To complete the pull request process, please assign sttts after the PR has been reviewed.
You can assign the PR to them by writing /assign @sttts in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[deads2k]

I think there's a reasonable case to made either way for spec or status.

I find the idea of placing this on the ClusterVersion as reasonable and probably better than a global configuration option.

/cc @openshift/team-cluster-operator

[wking]

ClusterProfile could also be a required field, but it might be an issue for updating a 4.6 cluster.

Yeah, optional defaulting to self-managed-high-availability is better than making it required.

I think there's a reasonable case to made either way for spec or status.

What's the case for spec? I think we want a status field, because component authors will need to put in a fair bit of work if cluster admins are allowed to flip between different profiles as a day-2 operation. So we should open with an immutable status property. If, in the future, users make a strong case for needing that sort of flexibility, we can grow an additional spec property, and the status property would become a reflection of the accepted spec setting which is currently being applied.

I think this discussion should be happening in an enhancement PR though, where the mechanism for initially specifying the profile is currently out of scope. I'd like a PR that updates the existing enhancement to remove that out-of-scope language and replace it with the ClusterVersion spec field. Or maybe a new, separate, follow-up enhancement, which you can link from the original enhancement. See also @LalatenduMohanty asking for an enhancement here.

Also on the table for the enhancement is passing environment variables around starting with the installer (openshift/installer#3986), which I'd be fine with too, as long as we were sure the choice was getting propagated up into Telemetry and Insights tarballs.

[deads2k]

What's the case for spec?

If we used a profile to indicate an intent like, "single node cluster" and have a goal of single node to HA, the value may need to change. I've heard some indication of that desire, but if we wanted to support it, spec would make sense.

I'm certainly fine with status, but I can see the case for spec. I can also see a path that starts with status and grows a spec if required.