rbac: allow build/image manipulation and retrieval

[cgwalters]

<walters> note: This drops 04df624#diff-879656d2f0921beeb4e9f00c109ade9a4fe9920cd742b43fa407144e6bbbd555R43 since I don't understand why it was there and it needs more motivation.

Splitting this out from the layering branch since it's one of the changes that currently conflicts on master. Prep for merging layering.

---

This give rbac permissions to the machine-config-controller service
account to manipulate Builds/BuildRequests and Imagestreams in the
machine-config-operator namespace, as well as push images into the internal registry.

This gives rbac permissions to the machine-config-daemon serviceaccount
to retrieve images from the internal registry.

This also updates the operator's sync so that these new registry role files will be
generated and deployed properly with the mco.

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cgwalters

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [cgwalters]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[jkyros]

• I was reading the secrets so I could get the actual docker pull secret for the machine-config-daemon so I could write it to /run/ostree/auth.json instead of using the service account token like I do in the controller.

• If I remember right, I did that because that's how @mkenigs was doing that in his test and it seemed like maybe that was what we wanted at the time?

• I really just need something that works as a working pull secret for the builtin registry so we can write it to /run/ostree/auth.json so rpm-ostree can pull the image out of that registry when it's time to rebase.

• I'd like to use the least invasive/least dangerous way to get it (and I 100% agree the current "read the namespace secrets" method is not the least invasive/least dangerous).

The service account token itself that is already available to the daemon does work if you do basic auth but I don't know how to format it for auth.json yet (just running "username:$token" through base64 doesn't seem to be good enough even though it seems like it should be.

Like, this will work:

podman pull image-registry.openshift-image-registry.svc:5000/openshift-machine-config-operator/whatever:latest --creds machine-config-daemon:$token

where $token is the contents of /var/run/secrets/kubernetes.io/serviceaccount/token)

It could also potentially be a completely different credential that we make specifically for this, if need be.

[cgwalters]

Right, hmm. Thinking about this more...I think this path is arguing a bit for the "special pod" path. That would mean that kubelet would be provided the necessary token to pull the image. But the "special pod" path has a lot of tradeoffs.

Here's another model - the operator passes the secret as part of the MCD pod instead. That seems like it'd Just Work, right?

[jkyros]

I see what you're saying about the "special pod".

I did manage to coax the serviceaccount token itself into docker credentials, but I'd rather not write the serviceaccount token to disk. I'd rather write a "useless" pull secret that can only pull.

I did initially want to just mount the pull secret -- the serviceaccount gets a default pull secret that works, but it's randomly named, so I can't reliably mount it. (If there is an easy way around the random secret naming thing I would love to know it).

[jkyros@jkyros-t590 cluster]$ oc describe serviceaccount machine-config-daemon
Name:                machine-config-daemon
Namespace:           openshift-machine-config-operator
Labels:              <none>
Annotations:         <none>
Image pull secrets:  machine-config-daemon-dockercfg-2w8kn
Mountable secrets:   machine-config-daemon-token-gpngr
                     machine-config-daemon-dockercfg-2w8kn
Tokens:              machine-config-daemon-token-58jwf
                     machine-config-daemon-token-gpngr
Events:              <none>

I was starting to go down the "we need to make a pull secret that has a reliable name and then have the pod mount it" path, and yes that should Just Work.

[cgwalters]

We talked about this real-time, I think the conclusion is that the operator (not the daemon) can load the serviceaccount/machine-config-daemon, find the image pull secret there, and mount it to the daemonset/machine-config-daemon pod.

[jkyros]

So I thought we had a good next step here, but I managed to hunt down where those automatic dockercfg secrets come from, and I don't know if this is "bad news", per se, but it really does look like openshift-controller-manager is doing the same thing (albeit more elegantly) that I was doing here:

https://github.com/jkyros/machine-config-operator/blob/b895594d51018c9e545655523ca3485f25fd9c52/pkg/daemon/daemon.go#L1717

by just "making a JSON wrapper" around the service account token:

https://github.com/openshift/openshift-controller-manager/blob/aca2e4f51451e7036e53e88c7f64c75c2a20fa3d/pkg/serviceaccounts/controllers/create_dockercfg_secrets.go#L468

I guess eventually if the daemon ends up having no real permissions, maybe it's okay to write that to disk, but right now that seems kind of...hmmm.

[cgwalters]

See internal thread "no auto generation of legacy service account token starting with 1.24"

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[openshift-merge-robot]

@cgwalters: PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-bot]

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

[openshift-ci]

@cgwalters: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-agnostic-upgrade	71d8df7	link	true	/test e2e-agnostic-upgrade
ci/prow/okd-scos-images	71d8df7	link	true	/test okd-scos-images

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[openshift-bot]

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

[openshift-ci]

@openshift-bot: Closed this PR.

Details

In response to this:

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.