Unable to Create Deployment for Memcached example using operator-sdk

[atef23]

Type of question

Setting up a new project with operator sdk version 0.19

Question

What did you do?
Created a new Memcached golang operator using the guide for the operator-sdk here:

https://sdk.operatorframework.io/docs/golang/quickstart/

and the stock controller from the operator-sdk source:
https://github.com/operator-framework/operator-sdk/blob/master/example/memcached-operator/memcached_controller.go.tmpl

What did you expect to see?
A deployment created upon applying the cr for the Memcached CRD.

What did you see instead? Under which circumstances?

The Memcached CRD gets created and a pod with the controller using my Memcached operator image gets created but when I attempt to apply the CR and create instances of the Memcached operator which should create deployment objects, I don't see any events or errors but no deployment gets created either.

Environment

• operator-sdk version:
  v0.19.0

• Kubernetes version information:

4.3.1

• Kubernetes cluster kind:

Additional context
I was able to get this working when creating an operator using the old scaffolding from operator-sdk v0.17

I see the instances of the Memcached operator but the controller doesn't create the deployment. Am I missing something with registering the controller with the manager or something?

Thanks,

Atef

[camilamacedo86]

Hi @atef,

Could you please run the following commands and add the output here?

• kubectl logs deployment.apps/memcached-operator-controller-manager -n memcached-operator-system -c manager
• kubectl get all
• kubectl get all -n memcached-operator-system

PS. I am guessing that you are looking for the CR deployment in the ns of the operator. If when you run kubectl apply -f config/sample/<cr-file> you did not inform the -n memcached-operator-system then, in OCP it will be created in the NS that you are kubectl get all will return that.

[atef23]

@camilamacedo86 thanks for providing those commands. here is the error I see repeatedly from the reconcile loop:

2020-07-29T01:12:22.390Z INFO controllers.Memcached Creating a new Deployment {"memcached": "memcached/memcached-sample", "Deployment.Namespace": "memcached", "Deployment.Name": "memcached-sample"} 2020-07-29T01:12:22.456Z ERROR controllers.Memcached Failed to create new Deployment {"memcached": "memcached/memcached-sample", "Deployment.Namespace": "memcached", "Deployment.Name": "memcached-sample", "error": "deployments.apps \"memcached-sample\" is forbidden: cannot set blockOwnerDeletion if an ownerReference refers to a resource you can't set finalizers on: , <nil>"} github.com/go-logr/zapr.(*zapLogger).Error /go/pkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128 github.com/example-inc/memcached-operator/controllers.(*MemcachedReconciler).Reconcile /workspace/controllers/memcached_controller.go:75 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.0/pkg/internal/controller/controller.go:256 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.0/pkg/internal/controller/controller.go:232 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.0/pkg/internal/controller/controller.go:211 k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1 /go/pkg/mod/k8s.io/apimachinery@v0.18.2/pkg/util/wait/wait.go:155 k8s.io/apimachinery/pkg/util/wait.BackoffUntil /go/pkg/mod/k8s.io/apimachinery@v0.18.2/pkg/util/wait/wait.go:156 k8s.io/apimachinery/pkg/util/wait.JitterUntil /go/pkg/mod/k8s.io/apimachinery@v0.18.2/pkg/util/wait/wait.go:133 k8s.io/apimachinery/pkg/util/wait.Until /go/pkg/mod/k8s.io/apimachinery@v0.18.2/pkg/util/wait/wait.go:90 2020-07-29T01:12:22.456Z ERROR controller-runtime.controller Reconciler error {"controller": "memcached", "request": "memcached/memcached-sample", "error": "deployments.apps \"memcached-sample\" is forbidden: cannot set blockOwnerDeletion if an ownerReference refers to a resource you can't set finalizers on: , <nil>"} github.com/go-logr/zapr.(*zapLogger).Error /go/pkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.0/pkg/internal/controller/controller.go:258 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.0/pkg/internal/controller/controller.go:232 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.0/pkg/internal/controller/controller.go:211 k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1 /go/pkg/mod/k8s.io/apimachinery@v0.18.2/pkg/util/wait/wait.go:155 k8s.io/apimachinery/pkg/util/wait.BackoffUntil /go/pkg/mod/k8s.io/apimachinery@v0.18.2/pkg/util/wait/wait.go:156 k8s.io/apimachinery/pkg/util/wait.JitterUntil /go/pkg/mod/k8s.io/apimachinery@v0.18.2/pkg/util/wait/wait.go:133 k8s.io/apimachinery/pkg/util/wait.Until /go/pkg/mod/k8s.io/apimachinery@v0.18.2/pkg/util/wait/wait.go:90

[atef23]

the output from oc get all:

`[atefaziz@localhost ~]$ oc get all
NAME READY STATUS RESTARTS AGE
pod/memcached-operator-controller-manager-59787dfdd4-88hn7 2/2 Running 0 68m

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/memcached-operator-controller-manager-metrics-service ClusterIP 172.30.150.61 8443/TCP 68m

NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/memcached-operator-controller-manager 1/1 1 1 68m

NAME DESIRED CURRENT READY AGE
replicaset.apps/memcached-operator-controller-manager-59787dfdd4 1 1 1 68m
`

[atef23]

@camilamacedo86 Here is my source for reference:

https://github.com/atef23/memcached-operator

I took the controller from the operator-sdk repository example controller:

https://github.com/atef23/memcached-operator/blob/master/controllers/memcached_controller.go

So I wonder if it's something related to RBAC policy?

[camilamacedo86]

Could you please try to add on top of your reconcile the RBAC marker:

// +kubebuilder:rbac:groups=apps,resources=deployments/finalizers,verbs=get;create;update;patch;delete

And then, run make manifests and build and push a new image and re-duo the test? Please, let us know if it worked for you.

[atef23]

@camilamacedo86 I updated that and then I get this error:

I0729 01:33:38.745704 1 request.go:621] Throttling request took 1.006031601s, request: GET:https://172.30.0.1:443/apis/autoscaling.openshift.io/v1?timeout=32s 2020-07-29T01:33:39.750Z INFO controller-runtime.metrics metrics server is starting to listen {"addr": "127.0.0.1:8080"} 2020-07-29T01:33:39.750Z INFO setup starting manager I0729 01:33:39.750919 1 leaderelection.go:242] attempting to acquire leader lease memcached2/f1c5ece8.example.com... 2020-07-29T01:33:39.750Z INFO controller-runtime.manager starting metrics server {"path": "/metrics"} I0729 01:33:39.843538 1 leaderelection.go:252] successfully acquired lease memcached2/f1c5ece8.example.com 2020-07-29T01:33:39.843Z INFO controller-runtime.controller Starting EventSource {"controller": "memcached", "source": "kind source: /, Kind="} 2020-07-29T01:33:39.843Z DEBUG controller-runtime.manager.events Normal {"object": {"kind":"ConfigMap","namespace":"memcached2","name":"f1c5ece8.example.com","uid":"d1dd9b99-85f1-41bf-bed3-6778b70a66d0","apiVersion":"v1","resourceVersion":"68902"}, "reason": "LeaderElection", "message": "memcached-operator-controller-manager-59787dfdd4-95ntg_b93a68fe-20f1-4a23-8430-7a284216c314 became leader"} 2020-07-29T01:33:39.943Z INFO controller-runtime.controller Starting EventSource {"controller": "memcached", "source": "kind source: /, Kind="} E0729 01:33:39.946783 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.2/tools/cache/reflector.go:125: Failed to list *v1.Deployment: deployments.apps is forbidden: User "system:serviceaccount:memcached2:default" cannot list resource "deployments" in API group "apps" at the cluster scope

Do I need to add a different clusterrole to the service account?

[atef23]

@camilamacedo86 I added this role to the manager-rolebinding:

roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin

Which works. Do I need to give it the cluster-admin role or can I get away with giving it less permissions?

[camilamacedo86]

HI @atef23,

The steps provide is expecting that the user logged has admin permission. See how to grant yourself cluster-admin privileges or be logged in as admin. (we need to clarify it in the docs)

The projects are now cluster-scoped by default which means that the logged user requires permission in the cluster to apply the cluster-roles. However, users still able to customize their projects to work with namespaced scope. See; https://master.sdk.operatorframework.io/docs/building-operators/golang/operator-scope/

[atef23]

@camilamacedo86 I’ll take a look there. thanks for your help with this. I’ll close this issue.

[camilamacedo86]

Let's keep this open for we make clear this point needs to be addressed in the docs.