BUILD-353: name changes, 2 CRDs instead of one for secrets and configmaps#55
Conversation
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: gabemontero The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
we have green e2e's on the latest rename @coreydaley @adambkaplan |
| Allowing the disabling processing of updates, or switching the default for the system as not dealing with | ||
| updates, but then allowing for opting into updates, is also under consideration. | ||
|
|
||
| //TODO do we punt on this with the separate CRDs now |
|
|
||
| - for the `Driver` string field, it needs to be ["csi-driver-projected-resource.openshift.io"](https://github.com/openshift/csi-driver-shared-resource/blob/1fcc354faa31f624086265ea2228661a0fc2e7b1/pkg/client/client.go#L28). | ||
| - for the `VolumeAttributes` map, this driver currently adds the "share" key (which maps the the `SharedResource` instance your `Pod` wants to use) in addition to the | ||
| - for the `VolumeAttributes` map, this driver currently adds the "share" key (which maps the the `SharedConfigMap` OR `SharedSecret` instance your `Pod` wants to use) in addition to the |
There was a problem hiding this comment.
| - for the `VolumeAttributes` map, this driver currently adds the "share" key (which maps the the `SharedConfigMap` OR `SharedSecret` instance your `Pod` wants to use) in addition to the | |
| - for the `VolumeAttributes` map, this driver currently adds the "share" key (which maps the `SharedConfigMap` OR `SharedSecret` instance your `Pod` wants to use) in addition to the |
There was a problem hiding this comment.
yeah actually the key name is not correct as well
it is sharedConfigMap or sharedSecret
I'll make that changes as well here
| oc apply -f ./deploy/0000_10_sharedconfigmap.crd.yaml | ||
| customresourcedefinition.apiextensions.k8s.io/sharedconfigmaps.sharedresource.openshift.io created | ||
| deploying hostpath components | ||
| ./deploy/0000_10_sharedsecrets.crd.yaml |
There was a problem hiding this comment.
| ./deploy/0000_10_sharedsecrets.crd.yaml | |
| ./deploy/0000_10_sharedsecret.crd.yaml |
| oc apply -f --filename https://raw.githubusercontent.com/openshift/csi-driver-shared-resource/release-4.10/deploy/00-namespace.yaml | ||
| oc apply -f --filename https://raw.githubusercontent.com/openshift/csi-driver-shared-resource/release-4.10/deploy/0000_10_sharedconfigmap.crd.yaml | ||
| oc apply -f --filename https://raw.githubusercontent.com/openshift/csi-driver-shared-resource/release-4.10/deploy/0000_10_sharedsecret.crd.yaml | ||
| oc apply -f --filename https://raw.githubusercontent.com/openshift/csi-driver-shared-resource/release-4.10/deploy/01-service-account.yaml | ||
| oc apply -f --filename https://raw.githubusercontent.com/openshift/csi-driver-shared-resource/release-4.10/deploy/02-cluster-role.yaml | ||
| oc apply -f --filename https://raw.githubusercontent.com/openshift/csi-driver-shared-resource/release-4.10/deploy/03-cluster-role-binding.yaml | ||
| oc apply -f --filename https://raw.githubusercontent.com/openshift/csi-driver-shared-resource/release-4.10/deploy/csi-hostpath-driverinfo.yaml | ||
| oc apply -f --filename https://raw.githubusercontent.com/openshift/csi-driver-shared-resource/release-4.10/deploy/csi-hostpath-plugin.yaml |
There was a problem hiding this comment.
Should we replace release-4.10 here with master instead?
There was a problem hiding this comment.
I don't think so, as master could be unstable, once a given release goes out.
There was a problem hiding this comment.
So we will have to update this readme every release?
There was a problem hiding this comment.
lesser of the evils IMO ... at least for now ... and if it lags, it is not the worse thing ... and it is easy enough for someone to figure our they can tweak that to another branch
possible this just goes away or morphs into a "dev only" mode at some point, in which case we switch to master
I'd rather hold off on that while we are in this intermediate state
| // | ||
| // spec: | ||
| // volumes: | ||
| // - name: shared-secret |
There was a problem hiding this comment.
| // - name: shared-secret | |
| // - name: shared-configmap |
| @@ -0,0 +1,73 @@ | |||
| package v1alpha1 | |||
There was a problem hiding this comment.
Should this file be named shared_configmap_type.go ?
There was a problem hiding this comment.
yeah and the secret one too
|
pushed updates from your comments as separate commits @coreydaley - thanks ! |
| rootCmd.Flags().AddGoFlagSet(flag.CommandLine) | ||
| rootCmd.Flags().StringVar(&endPoint, "endpoint", "unix://tmp/csi.sock", "CSI endpoint") | ||
| rootCmd.Flags().StringVar(&driverName, "drivername", string(v1.SharedResourcesCSIDriver), "name of the driver") | ||
| //rootCmd.Flags().StringVar(&driverName, "drivername", string(v1.SharedResourcesCSIDriver), "name of the driver") |
There was a problem hiding this comment.
Do we still need this commented out line?
There was a problem hiding this comment.
I'd like to leave it as per the TODO comment below we'll swtich back to it once you've updated that driver name with your openshift/api PR
| role.rbac.authorization.k8s.io/shared-resource-my-share created | ||
| rolebinding.rbac.authorization.k8s.io/shared-resource-my-share created | ||
| share.sharedresource.storage.openshift.io/my-share created | ||
| sharedconfig.sharedresource.openshift.io/my-share created |
| ```bash | ||
| $ oc get events | ||
| 0s Warning FailedMount pod/my-csi-app MountVolume.SetUp failed for volume "my-csi-volume" : rpc error: code = InvalidArgument desc = the csi driver volumeAttribute 'share' reference had an error: sharedresource.storage.openshift.io "my-share" not found | ||
| 0s Warning FailedMount pod/my-csi-app MountVolume.SetUp failed for volume "my-csi-volume" : rpc error: code = InvalidArgument desc = the csi driver volumeAttribute 'share' reference had an error: sharedconfig.sharedresource.openshift.io "my-share" not found |
196638c to
9b75750
Compare
|
ok @coreydaley squashed the previous review commits, add a new one for the next round of corrections from you - thanks ! |
| @@ -0,0 +1,22 @@ | |||
| # this is the boilerplate crd def that controller-gen reads and modifies with the | |||
| # contents from share_secret_type.go | |||
There was a problem hiding this comment.
| # contents from share_secret_type.go | |
| # contents from shared_secret_type.go |
| @@ -0,0 +1,104 @@ | |||
| # this is the boilerplate crd def that controller-gen reads and modifies with the | |||
| # contents from share_configmap_type.go | |||
There was a problem hiding this comment.
| # contents from share_configmap_type.go | |
| # contents from shared_configmap_type.go |
| @@ -1,27 +1,27 @@ | |||
| # this is the boilerplate crd def that controller-gen reads and modifies with the | |||
| # contents from share_type.go | |||
| # contents from share_secret_type.go | |||
There was a problem hiding this comment.
| # contents from share_secret_type.go | |
| # contents from shared_secret_type.go |
| // +genclient:nonNamespaced | ||
| // +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object | ||
|
|
||
| // SharedSecret allows a Secret o be shared across namespaces. |
There was a problem hiding this comment.
| // SharedSecret allows a Secret o be shared across namespaces. | |
| // SharedSecret allows a Secret to be shared across namespaces. |
| // sharedConfigMap: my-share | ||
| // | ||
| // For the mount to be successful, the pod's service account must be granted permission to 'use' the named SharedConfigMap object | ||
| // within its namespace with an appropriate Role and RoleBinding. For compactness, here are example `oc` invocations for creating |
There was a problem hiding this comment.
| // within its namespace with an appropriate Role and RoleBinding. For compactness, here are example `oc` invocations for creating | |
| // within its namespace with an appropriate Role and RoleBinding. For compactness, here are example `oc` invocations for creating |
| secretsWithShare = sync.Map{} | ||
| // sharesWaitingOnSecrets conversely is for when a share has been created that references a secret, but that | ||
| // secret has not been recognized by the controller; quite possibly timing events on when we learn of shares | ||
| // secret has not been recognized by the controller; quite possibly timing events on when we learn of shareConfigMaps |
There was a problem hiding this comment.
| // secret has not been recognized by the controller; quite possibly timing events on when we learn of shareConfigMaps | |
| // secret has not been recognized by the controller; quite possibly timing events on when we learn of sharedSecrets |
| secretKey := BuildKey(br) | ||
| secretsWithShare.Store(secretKey, secret) | ||
| //NOTE: share update ranger will store share in shares sync.Map | ||
| //NOTE: share update ranger will store share in shareConfigMaps sync.Map |
There was a problem hiding this comment.
| //NOTE: share update ranger will store share in shareConfigMaps sync.Map | |
| //NOTE: share update ranger will store share in shareSecrets sync.Map |
| //NOTE: share update ranger will store share in shareConfigMaps sync.Map | ||
| // and we are supplying only this specific share to the csi driver update range callbacks. | ||
| shareUpdateCallbacks.Range(buildRanger(buildCallbackMap(share.Name, share))) | ||
| shareConfigMapsUpdateCallbacks.Range(buildRanger(buildCallbackMap(share.Name, share))) |
There was a problem hiding this comment.
| shareConfigMapsUpdateCallbacks.Range(buildRanger(buildCallbackMap(share.Name, share))) | |
| shareSecretsUpdateCallbacks.Range(buildRanger(buildCallbackMap(share.Name, share))) |
| func RegisterSecretUpsertCallback(volID string, f func(key, value interface{}) bool) { | ||
| secretUpsertCallbacks.Store(volID, f) | ||
| // cycle through the secrets with shares, where if the share associated with the volID CSI volume mount references | ||
| // cycle through the secrets with shareConfigMaps, where if the share associated with the volID CSI volume mount references |
There was a problem hiding this comment.
| // cycle through the secrets with shareConfigMaps, where if the share associated with the volID CSI volume mount references | |
| // cycle through the secrets with sharedSecrets, where if the share associated with the volID CSI volume mount references |
| func AddSharedSecret(share *sharev1alpha1.SharedSecret) { | ||
| br := share.Spec.Secret | ||
| key := BuildKey(br) | ||
| klog.V(4).Infof("AddSharedConfigMap key %s", key) |
There was a problem hiding this comment.
| klog.V(4).Infof("AddSharedConfigMap key %s", key) | |
| klog.V(4).Infof("AddSharedSecret key %s", key) |
|
|
||
| c.informerFactory.Start(stopCh) | ||
| c.shareInformerFactory.Start(stopCh) | ||
| c.sharedConfigMapInformerFactory.Start(stopCh) |
There was a problem hiding this comment.
Maybe I missed it, but I don't see where you are starting the sharedSecretInformerFactory?
There was a problem hiding this comment.
yep I missed it - thanks
| Verb: verb, | ||
| } | ||
| c.shareWorkqueue.Add(event) | ||
| c.sharedConfigMapWorkqueue.Add(event) |
There was a problem hiding this comment.
| c.sharedConfigMapWorkqueue.Add(event) | |
| c.sharedSecretWorkqueue.Add(event) |
| func (c *Controller) sharedSecretEventProcessor() { | ||
| for { | ||
| obj, shutdown := c.shareWorkqueue.Get() | ||
| obj, shutdown := c.sharedConfigMapWorkqueue.Get() |
There was a problem hiding this comment.
| obj, shutdown := c.sharedConfigMapWorkqueue.Get() | |
| obj, shutdown := c.sharedSecretMapWorkqueue.Get() |
|
|
||
| func() { | ||
| defer c.shareWorkqueue.Done(obj) | ||
| defer c.sharedConfigMapWorkqueue.Done(obj) |
There was a problem hiding this comment.
| defer c.sharedConfigMapWorkqueue.Done(obj) | |
| defer c.sharedSecretWorkqueue.Done(obj) |
| event, ok := obj.(client.Event) | ||
| if !ok { | ||
| c.shareWorkqueue.Forget(obj) | ||
| c.sharedConfigMapWorkqueue.Forget(obj) |
There was a problem hiding this comment.
| c.sharedConfigMapWorkqueue.Forget(obj) | |
| c.sharedSecretWorkqueue.Forget(obj) |
| sShare, err = client.GetListers().SharedSecrets.Get(secretShareName) | ||
| if err != nil { | ||
| return nil, nil, status.Errorf(codes.InvalidArgument, | ||
| "the csi driver volumeAttribute '%s' reference had an error: %s", secretShareName, err.Error()) |
There was a problem hiding this comment.
| "the csi driver volumeAttribute '%s' reference had an error: %s", secretShareName, err.Error()) | |
| "the csi driver volumeAttribute %q reference had an error: %s", secretShareName, err.Error()) |
| if cmShare != nil { | ||
| if len(strings.TrimSpace(cmShare.Spec.ConfigMap.Namespace)) == 0 { | ||
| return nil, nil, status.Errorf(codes.InvalidArgument, | ||
| "the SharedConfigMap %s backing resource namespace needs to be set", configMapShareName) |
There was a problem hiding this comment.
| "the SharedConfigMap %s backing resource namespace needs to be set", configMapShareName) | |
| "the SharedConfigMap %q backing resource namespace needs to be set", configMapShareName) |
| } | ||
| if len(strings.TrimSpace(cmShare.Spec.ConfigMap.Name)) == 0 { | ||
| return nil, nil, status.Errorf(codes.InvalidArgument, | ||
| "the SharedConfigMap %s backing resource name needs to be set", configMapShareName) |
There was a problem hiding this comment.
| "the SharedConfigMap %s backing resource name needs to be set", configMapShareName) | |
| "the SharedConfigMap %q backing resource name needs to be set", configMapShareName) |
| kind = sharev1alpha1.ResourceReferenceTypeSecret | ||
| if len(strings.TrimSpace(sShare.Spec.Secret.Namespace)) == 0 { | ||
| return nil, nil, status.Errorf(codes.InvalidArgument, | ||
| "the SharedSecret %s backing resource namespace needs to be set", secretShareName) |
There was a problem hiding this comment.
| "the SharedSecret %s backing resource namespace needs to be set", secretShareName) | |
| "the SharedSecret %q backing resource namespace needs to be set", secretShareName) |
| } | ||
| if len(strings.TrimSpace(sShare.Spec.Secret.Name)) == 0 { | ||
| return nil, nil, status.Errorf(codes.InvalidArgument, | ||
| "the SharedSecret %s backing resource name needs to be set", secretShareName) |
There was a problem hiding this comment.
| "the SharedSecret %s backing resource name needs to be set", secretShareName) | |
| "the SharedSecret %q backing resource name needs to be set", secretShareName) |
8ff140a to
a8d44b3
Compare
|
ok @coreydaley I believe I have addressed the latest set of comments in 2 "core-3" commits (where I squashed the previous review commits unit tests pass for me locally but I'm still waiting on my cluster to come up to try the e2e's I'll run those locally when the cluster comes up if I see any failures here in the PR |
|
Should we update the corresponding enhancement proposal with this change ? |
yes I'm including that as one of the to-do's for BUILD-353 just getting the code ready first |
|
aws error with slow /test e2e-aws-csi-driver-slow |
|
ok the e2e's are passing for me locally ... would anticipate the slow e2e passing if it gets past unrelated flakes |
|
and the e2e's are green as well @coreydaley assuming there is nothing else, and @adambkaplan deferral to us in scrum/office hours today, I'll squash and then we can lgtm / merge And you can start mapping the api changes back to openshift/api#979 |
coreydaley
left a comment
There was a problem hiding this comment.
I think these are the last two issues.
If you want to squash your commits and then I think we can get this merged and I can proceed with the openshift/api stuff.
| or namespace administrator. | ||
|
|
||
| The full definition of the `SharedResource` custom resource can be found [here](deploy/0000_10_sharedresource.crd.yaml). | ||
| The full definition of the `SharedConfigMap` can be found [here](deploy/0000_10_sharedconfigmap.crd.yaml)or `SharedSecret` custom resource can be found [here](deploy/0000_10_sharedsecret.crd.yaml). |
There was a problem hiding this comment.
| The full definition of the `SharedConfigMap` can be found [here](deploy/0000_10_sharedconfigmap.crd.yaml)or `SharedSecret` custom resource can be found [here](deploy/0000_10_sharedsecret.crd.yaml). | |
| The full definition of the `SharedConfigMap` can be found [here](deploy/0000_10_sharedconfigmap.crd.yaml) or `SharedSecret` custom resource can be found [here](deploy/0000_10_sharedsecret.crd.yaml). |
| c.informerFactory.Start(stopCh) | ||
| c.shareInformerFactory.Start(stopCh) | ||
| c.sharedConfigMapInformerFactory.Start(stopCh) | ||
| c.sharedConfigMapInformerFactory.Start(stopCh) |
There was a problem hiding this comment.
| c.sharedConfigMapInformerFactory.Start(stopCh) | |
| c.sharedSecretInformerFactory.Start(stopCh) |
doc updates update simple example controller updates driver rename add SharedSecret and SharedConfigMap generate-crd e2e tests, including replacing any configmap to secret change tests
a8d44b3 to
7a51ab2
Compare
|
corrections pulled in and pushed, commits squashed @coreydaley thanks !! |
|
/lgtm |
in repro API should be drag and drop back to openshift/api
the 2 CRDs resulted in removing support for switching between configmaps and secrets for a "share" (a good thing IMO in hindsight)
/assign @coreydaley
/assign @adambkaplan