From ebd94a2547e7d1e64a7f5407b4ce0a84f01aff4c Mon Sep 17 00:00:00 2001 From: Camila Macedo Date: Sun, 26 Jul 2020 19:30:20 +0100 Subject: [PATCH 1/3] doc: quickstart for go projects --- .../golang/advanced-topics.md | 2 +- .../building-operators/golang/crds-scope.md | 4 +- .../golang/operator-scope.md | 2 +- .../golang/project_migration_guide.md | 6 +- .../building-operators/golang/quickstart.md | 482 ++---------------- .../building-operators/golang/tutorial.md | 450 ++++++++++++++++ .../building-operators/golang/webhooks.md | 13 +- 7 files changed, 513 insertions(+), 446 deletions(-) create mode 100644 website/content/en/docs/building-operators/golang/tutorial.md diff --git a/website/content/en/docs/building-operators/golang/advanced-topics.md b/website/content/en/docs/building-operators/golang/advanced-topics.md index 4fe2fde316..1b89784d7c 100644 --- a/website/content/en/docs/building-operators/golang/advanced-topics.md +++ b/website/content/en/docs/building-operators/golang/advanced-topics.md @@ -1,7 +1,7 @@ --- title: Advanced Topics linkTitle: Advanced Topics -weight: 50 +weight: 70 --- ### Manage CR status conditions diff --git a/website/content/en/docs/building-operators/golang/crds-scope.md b/website/content/en/docs/building-operators/golang/crds-scope.md index 480ba48ed2..6e85b58167 100644 --- a/website/content/en/docs/building-operators/golang/crds-scope.md +++ b/website/content/en/docs/building-operators/golang/crds-scope.md @@ -1,7 +1,7 @@ --- title: CRD Scope linkTitle: CRD Scope -weight: 30 +weight: 60 --- ## Overview @@ -97,5 +97,5 @@ spec: ``` [RBAC]: https://kubernetes.io/docs/reference/access-authn-authz/rbac/ -[manager_user_guide]:/docs/building-operators/golang/quickstart/#manager +[manager_user_guide]:/docs/building-operators/golang/tutorial/#manager [manager_options]: https://godoc.org/github.com/kubernetes-sigs/controller-runtime/pkg/manager#Options diff --git a/website/content/en/docs/building-operators/golang/operator-scope.md b/website/content/en/docs/building-operators/golang/operator-scope.md index 16395d440d..c4ef2e1feb 100644 --- a/website/content/en/docs/building-operators/golang/operator-scope.md +++ b/website/content/en/docs/building-operators/golang/operator-scope.md @@ -1,7 +1,7 @@ --- title: Operators Scope linkTitle: Operator Scope -weight: 20 +weight: 50 --- ## Overview diff --git a/website/content/en/docs/building-operators/golang/project_migration_guide.md b/website/content/en/docs/building-operators/golang/project_migration_guide.md index bcf88d5e13..8828ae7103 100644 --- a/website/content/en/docs/building-operators/golang/project_migration_guide.md +++ b/website/content/en/docs/building-operators/golang/project_migration_guide.md @@ -159,10 +159,10 @@ func (r *MemcachedReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error) { **Note**: To update `config/rbac/role.yaml` after changing the markers, run `make manifests`. -The project can now be built, and the operator can be deployed on-cluster. For further steps regarding the deployment of the operator, creation of custom resources and cleaning up of resources, refer to the [quickstart guide][kb_quickstart]. +The project can now be built, and the operator can be deployed on-cluster. For further steps regarding the deployment of the operator, creation of custom resources and cleaning up of resources, refer to the [quickstart guide][kb_tutorial]. -[memcached-operator]:/docs/building-operators/golang/quickstart/ +[memcached-operator]:/docs/building-operators/golang/tutorial/ [git_tool]: https://git-scm.com/downloads [go_tool]: https://golang.org/dl/ [docker_tool]:https://docs.docker.com/install/ @@ -177,6 +177,6 @@ The project can now be built, and the operator can be deployed on-cluster. For f [memcached_cr]: https://github.com/operator-framework/operator-sdk-samples/blob/master/go/memcached-operator/deploy/crds/cache.example.com_v1alpha1_memcached_cr.yaml [memcached_types]: https://github.com/operator-framework/operator-sdk-samples/blob/master/go/memcached-operator/pkg/apis/cache/v1alpha1/memcached_types.go [kb_memcached_controller]: https://github.com/operator-framework/operator-sdk/blob/master/example/memcached-operator/memcached_controller.go.tmpl -[kb_quickstart]: /docs/building-operators/golang/quickstart/ +[kb_tutorial]: /docs/building-operators/golang/tutorial/ [install_guide]: /docs/installation/install-operator-sdk/ [env-test-setup]: /docs/building-operators/golang/references/env-test-setup \ No newline at end of file diff --git a/website/content/en/docs/building-operators/golang/quickstart.md b/website/content/en/docs/building-operators/golang/quickstart.md index 87de194ab8..c66cab35fc 100644 --- a/website/content/en/docs/building-operators/golang/quickstart.md +++ b/website/content/en/docs/building-operators/golang/quickstart.md @@ -1,502 +1,120 @@ --- -title: Golang Based Operator Quickstart -linkTitle: Quickstart -weight: 10 +title: Golang Based Operator QuickStart +linkTitle: QuickStart +weight: 20 --- -**NOTE:** For the SDK versions prior to `v0.19.0` please consult the [legacy docs][legacy_quickstart_doc] for the [legacy CLI][legacy_CLI] and project. - -This guide walks through an example of building a simple memcached-operator using the operator-sdk CLI tool and controller-runtime library API. - ## Prerequisites - [go][go_tool] version v1.13+. - [docker][docker_tool] version 17.03+. - [kubectl][kubectl_tool] version v1.11.3+. -- [kustomize][kustomize_tool] v3.1.0+ +- [operator-sdk][operator_install] v.0.19+ - Access to a Kubernetes v1.11.3+ cluster. -## Create a new project - -Use the CLI to create a new memcached-operator project: +## Creating a Project +Create a directory, and then run the init command inside of it to generate a new project. + ```sh -$ mkdir -p $HOME/projects/memcached-operator -$ cd $HOME/projects/memcached-operator -# we'll use a domain of example.com -# so all API groups will be .example.com -$ operator-sdk init --domain=example.com --repo=github.com/example-inc/memcached-operator -``` - -To learn about the project directory structure, see [Kubebuilder project layout][kubebuilder_layout_doc] doc. - -#### A note on dependency management - -`operator-sdk init` generates a `go.mod` file to be used with [Go modules][go_mod_wiki]. The `--repo=` flag is required when creating a project outside of `$GOPATH/src`, as scaffolded files require a valid module path. Ensure you [activate module support][activate_modules] by running `export GO111MODULE=on` before using the SDK. - -### Manager -The main program for the operator `main.go` initializes and runs the [Manager][manager_go_doc]. - -See the [Kubebuilder entrypoint doc][kubebuilder_entrypoint_doc] for more details on how the manager registers the Scheme for the custom resource API defintions, and sets up and runs controllers and webhooks. - - -The Manager can restrict the namespace that all controllers will watch for resources: -```Go -mgr, err := manager.New(cfg, manager.Options{Namespace: namespace}) -``` -By default this will be the namespace that the operator is running in. To watch all namespaces leave the namespace option empty: -```Go -mgr, err := manager.New(cfg, manager.Options{Namespace: ""}) -``` - -It is also possible to use the [MultiNamespacedCacheBuilder][multi-namespaced-cache-builder] to watch a specific set of namespaces: -```Go -var namespaces []string // List of Namespaces -// Create a new Cmd to provide shared dependencies and start components -mgr, err := manager.New(cfg, manager.Options{ - NewCache: cache.MultiNamespacedCacheBuilder(namespaces), -}) -``` - -#### Operator scope - -Read the [operator scope][operator_scope] documentation on how to run your operator as namespace-scoped vs cluster-scoped. - -By default the main program will set the manager's namespace using the value of `WATCH_NAMESPACE` env defined in `deploy/operator.yaml`. - -### Multi-Group APIs - -Before creating an API and controller, consider if your operator's API requires multiple [groups][API-groups]. -If yes then add the line `multigroup: true` in the `PROJECT` file which should look like the following: - -```YAML -domain: example.com -layout: go.kubebuilder.io/v2 -multigroup: true -... +$ mkdir $GOPATH/src/memcached-operator +$ cd $GOPATH/src/memcached-operator +$ operator-sdk init ``` -For multi-group projects, the API Go type files are created under `apis///` and the controllers under `controllers//`. - -This guide will cover the default case of a single group API. - -## Create a new API and Controller - -Create a new Custom Resource Definition(CRD) API with group `cache` version `v1alpha1` and Kind Memcached. -When prompted, enter yes `y` for creating both the resource and controller. - -```console -$ operator-sdk create api --group=cache --version=v1alpha1 --kind=Memcached -Create Resource [y/n] -y -Create Controller [y/n] -y -Writing scaffold for you to edit... -api/v1alpha1/memcached_types.go -controllers/memcached_controller.go -... -``` - -This will scaffold the Memcached resource API at `api/v1alpha1/memcached_types.go` and the controller at `controllers/memcached_controller.go`. - -See the [API terminology doc][api_terms_doc] for details on the CRD API conventions. - -To understand the API Go types and controller scaffolding see the Kubebuilder [api doc][kb_api_doc] and [controller doc][kb_controller_doc]. - -### Define the API -Define the API for the Memcached Custom Resource(CR) by modifying the Go type definitions at `api/v1alpha1/memcached_types.go` to have the following spec and status: +## Creating an API -```Go -// MemcachedSpec defines the desired state of Memcached -type MemcachedSpec struct { - // +kubebuilder:validation:Minimum=0 - // Size is the size of the memcached deployment - Size int32 `json:"size"` -} - -// MemcachedStatus defines the observed state of Memcached -type MemcachedStatus struct { - // Nodes are the names of the memcached pods - Nodes []string `json:"nodes"` -} -``` - -Add the `+kubebuilder:subresource:status` [marker][status_marker] to add a [status subresource][status_subresource] to the CRD manifest so that the controller can update the CR status without changing the rest of the CR object: - -```Go -// Memcached is the Schema for the memcacheds API -// +kubebuilder:subresource:status -type Memcached struct { - metav1.TypeMeta `json:",inline"` - metav1.ObjectMeta `json:"metadata,omitempty"` - - Spec MemcachedSpec `json:"spec,omitempty"` - Status MemcachedStatus `json:"status,omitempty"` -} -``` - -After modifying the `*_types.go` file always run the following command to update the generated code for that resource type: +Let's create a new API with `(group/version)` as `cache/v1` and new Kind(CRD) Memcached on it: ```sh -$ make generate +$ operator-sdk create api --group cache --version v1 --kind Memcached ``` -The above makefile target will invoke the [controller-gen][controller_tools] utility to update the `api/v1alpha1/zz_generated.deepcopy.go` file to ensure our API's Go type definitons implement the `runtime.Object` interface that all Kind types must implement. - -### Generating CRD manifests - -Once the API is defined with spec/status fields and CRD validation markers, the CRD manifests can be generated and updated with the following command: - -```console -$ make manifests -``` - -This makefile target will invoke controller-gen to generate the CRD manifests at `config/crd/bases/cache.example.com_memcacheds.yaml`. - -#### OpenAPI validation - -OpenAPIv3 schemas are added to CRD manifests in the `spec.validation` block when the manifests are generated. This validation block allows Kubernetes to validate the properties in a Memcached Custom Resource when it is created or updated. - -Markers (annotations) are available to configure validations for your API. These markers will always have a `+kubebuilder:validation` prefix. - -Usage of markers in API code is discussed in the kubebuilder [CRD generation][generating-crd] and [marker][markers] documentation. A full list of OpenAPIv3 validation markers can be found [here][crd-markers]. - -To learn more about OpenAPI v3.0 validation schemas in CRDs, refer to the [Kubernetes Documentation][doc-validation-schema]. - -### Implement the Controller - -For this example replace the generated controller file `controllers/memcached_controller.go` with the example [`memcached_controller.go`][memcached_controller] implementation. - -The example controller executes the following reconciliation logic for each Memcached CR: -- Create a memcached Deployment if it doesn't exist -- Ensure that the Deployment size is the same as specified by the Memcached CR spec -- Update the Memcached CR status using the status writer with the names of the memcached pods - -The next two subsections explain how the controller watches resources and how the reconcile loop is triggered. Skip to the [Build](#build-and-run-the-operator) section to see how to build and run the operator. - -#### Resources watched by the Controller - -The `SetupWithManager()` function in `controllers/memcached_controller.go` specifies how the controller is built to watch a CR and other resources that are owned and managed by that controller. - -```Go -func (r *MemcachedReconciler) SetupWithManager(mgr ctrl.Manager) error { - return ctrl.NewControllerManagedBy(mgr). - For(&cachev1alpha1.Memcached{}). - Owns(&appsv1.Deployment{}). - Complete(r) -} -``` - -The `NewControllerManagedBy()` provides a controller builder that allows various controller configurations. - -`For(&cachev1alpha1.Memcached{})` specifies the Memcached type as the primary resource to watch. For each Memcached type Add/Update/Delete event the reconcile loop will be sent a reconcile `Request` (a namespace/name key) for that Memcached object. - -`Owns(&appsv1.Deployment{})` specifies the Deployments type as the secondary resource to watch. For each Deployment type Add/Update/Delete event, the event handler will map each event to a reconcile `Request` for the owner of the Deployment. Which in this case is the Memcached object for which the Deployment was created. - -#### Controller Configurations +**Note** If you press `y` for Create Resource `[y/n]` and for Create Controller `[y/n]` then this will create the files `api/v1/memcached_types.go` where the API is defined and the `controllers/memcached_controller.go` where the reconciliation business logic will be done for the `Memcached` Kind(CRD). -There are a number of other useful configurations that can be made when initialzing a controller. For more details on these configurations consult the upstream [builder][builder_godocs] and [controller][controller_godocs] godocs. +## Applying the CRDs into the cluster: -- Set the max number of concurrent Reconciles for the controller via the [`MaxConcurrentReconciles`][controller_options] option. Defaults to 1. - ```Go - func (r *MemcachedReconciler) SetupWithManager(mgr ctrl.Manager) error { - return ctrl.NewControllerManagedBy(mgr). - For(&cachev1alpha1.Memcached{}). - Owns(&appsv1.Deployment{}). - WithOptions(controller.Options{ - MaxConcurrentReconciles: 2, - }). - Complete(r) - } - ``` -- Filter watch events using [predicates][event_filtering] -- Choose the type of [EventHandler][event_handler_godocs] to change how a watch event will translate to reconcile requests for the reconcile loop. For operator relationships that are more complex than primary and secondary resources, the [`EnqueueRequestsFromMapFunc`][enqueue_requests_from_map_func] handler can be used to transform a watch event into an arbitrary set of reconcile requests. - - -#### Reconcile loop - -Every Controller has a Reconciler object with a `Reconcile()` method that implements the reconcile loop. The reconcile loop is passed the [`Request`][request-go-doc] argument which is a Namespace/Name key used to lookup the primary resource object, Memcached, from the cache: - -```Go -import ( - ctrl "sigs.k8s.io/controller-runtime" - - cachev1alpha1 "github.com/example-inc/memcached-operator/api/v1alpha1" - ... -) - -func (r *MemcachedReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error) { - // Lookup the Memcached instance for this reconcile request - memcached := &cachev1alpha1.Memcached{} - err := r.Get(ctx, req.NamespacedName, memcached) - ... -} -``` - -Based on the return values, [`Result`][result_go_doc] and error, the `Request` may be requeued and the reconcile loop may be triggered again: - -```Go -// Reconcile successful - don't requeue -return ctrl.Result{}, nil -// Reconcile failed due to error - requeue -return ctrl.Result{}, err -// Requeue for any reason other than an error -return ctrl.Result{Requeue: true}, nil -``` - -You can set the `Result.RequeueAfter` to requeue the `Request` after a grace period as well: -```Go -import "time" - -// Reconcile for any reason other than an error after 5 seconds -return ctrl.Result{RequeueAfter: time.Second*5}, nil -``` - -**Note:** Returning `Result` with `RequeueAfter` set is how you can periodically reconcile a CR. - -For a guide on Reconcilers, Clients, and interacting with resource Events, see the [Client API doc][doc_client_api]. - -### Specify permissions and generate RBAC manifests - -The controller needs certain RBAC permissions to interact with the resources it manages. These are specified via [RBAC markers][rbac_markers] like the following: - -```Go -// +kubebuilder:rbac:groups=cache.example.com,resources=memcacheds,verbs=get;list;watch;create;update;patch;delete -// +kubebuilder:rbac:groups=cache.example.com,resources=memcacheds/status,verbs=get;update;patch -// +kubebuilder:rbac:groups=apps,resources=deployments,verbs=get;list;watch;create;update;patch;delete -// +kubebuilder:rbac:groups=core,resources=pods,verbs=get;list; - -func (r *MemcachedReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error) { -``` - -The `ClusterRole` manifest at `config/rbac/role.yaml` is generated from the above markers via controller-gen with the following command: +To apply the `Memcached` Kind(CRD): ```sh -$ make manifests +$ make install ``` -## Build and run the operator +## Running it locally -Before running the operator, the CRD must be registered with the Kubernetes apiserver: +To run the project out of the cluster: ```sh -$ make install +$ make run ``` -Once this is done, there are two ways to run the operator: - -- As Go program outside a cluster -- As a Deployment inside a Kubernetes cluster - ## Configuring your test environment -Projects are scaffolded with unit tests that utilize the [envtest](https://godoc.org/sigs.k8s.io/controller-runtime/pkg/envtest) -library, which requires certain Kubernetes server binaries be present locally. -Installation instructions can be found [here][env-test-setup]. - -### 1. Run locally outside the cluster - -To run the operator locally execute the following command: +Projects are scaffolded with tests that requires certain Kubernetes server binaries be present locally. Run: ```sh -$ make run ENABLE_WEBHOOKS=false +$ curl -sSLo setup_envtest.sh https://raw.githubusercontent.com/kubernetes-sigs/kubebuilder/master/scripts/setup_envtest_bins.sh +$ chmod +x setup_envtest.sh +$ ./setup_envtest.sh v1.18.2 v3.4.3 ``` -### 2. Run as a Deployment inside the cluster - -#### Build and push the image - -Before building the operator image, ensure the generated Dockerfile references -the base image you want. You can change the default "runner" image `gcr.io/distroless/static:nonroot` -by replacing its tag with another, for example `alpine:latest`, and removing -the `USER: nonroot:nonroot` directive. +**Note:** More info can be found [here][env-test-setup]. -To build and push the operator image, use the following `make` commands. -Make sure to modify the `IMG` arg in the example below to reference a container repository that -you have access to. You can obtain an account for storing containers at -repository sites such quay.io or hub.docker.com. This example uses quay. +## Building and Pushing the Project Image -Build the image: -```sh -$ export USERNAME= - -$ make docker-build IMG=quay.io/$USERNAME/memcached-operator:v0.0.1 -``` - -Push the image to a repository: +To build and push your image to your repository : ```sh -$ make docker-push IMG=quay.io/$USERNAME/memcached-operator:v0.0.1 +$ make docker-build docker-push IMG=/:tag ``` -**Note**: -The name and tag of the image (`IMG=/:tag`) in both the commands can also be set in the Makefile. Modify the line which has `IMG ?= controller:latest` to set your desired default image name. +**Note** To allow the cluster pull the image the repository needs to be set as public. -#### Deploy the operator - -For this example we will run the operator in the `default` namespace which can be specified for all resources in `config/default/kustomization.yaml`: - -```sh -$ cd config/default/ && kustomize edit set namespace "default" && cd ../.. -``` +## Running it on Cluster -Run the following to deploy the operator. This will also install the RBAC manifests from `config/rbac`. +Deploy the project to the cluster: ```sh -$ make deploy IMG=quay.io/$USERNAME/memcached-operator:v0.0.1 +$ make deploy IMG=/:tag ``` -*NOTE* If you have enabled webhooks in your deployments, you will need to have cert-manager already installed -in the cluster or `make deploy` will fail when creating the cert-manager resources. +## Applying the CR's into the cluster: -Verify that the memcached-operator is up and running: - -```console -$ kubectl get deployment -NAME READY UP-TO-DATE AVAILABLE AGE -memcached-operator-controller-manager 1/1 1 1 8m -``` - -### 3. Deploy your Operator with the Operator Lifecycle Manager (OLM) - -OLM will manage creation of most if not all resources required to run your operator, using a bit of setup from other `operator-sdk` commands. Check out the [docs][cli-run-olm] for more information. - -## Create a Memcached CR - -Update the sample Memcached CR manifest at `config/samples/cache_v1alpha1_memcached.yaml` and define the `spec` as the following: - -```YAML -apiVersion: cache.example.com/v1alpha1 -kind: Memcached -metadata: - name: memcached-sample -spec: - size: 3 -``` - -Create the CR: +To create instances (CR's) of the `Memcached` Kind (CRD) in the same namespaced of the operator ```sh -$ kubectl apply -f config/samples/cache_v1alpha1_memcached.yaml -``` - -Ensure that the memcached operator creates the deployment for the sample CR with the correct size: - -```console -$ kubectl get deployment -NAME READY UP-TO-DATE AVAILABLE AGE -memcached-operator-controller-manager 1/1 1 1 8m -memcached-sample 3/3 3 3 1m -``` - -Check the pods and CR status to confirm the status is updated with the memcached pod names: - -```console -$ kubectl get pods -NAME READY STATUS RESTARTS AGE -memcached-sample-6fd7c98d8-7dqdr 1/1 Running 0 1m -memcached-sample-6fd7c98d8-g5k7v 1/1 Running 0 1m -memcached-sample-6fd7c98d8-m7vn7 1/1 Running 0 1m -``` - -```console -$ kubectl get memcached/memcached-sample -o yaml -apiVersion: cache.example.com/v1alpha1 -kind: Memcached -metadata: - clusterName: "" - creationTimestamp: 2018-03-31T22:51:08Z - generation: 0 - name: memcached-sample - namespace: default - resourceVersion: "245453" - selfLink: /apis/cache.example.com/v1alpha1/namespaces/default/memcacheds/memcached-sample - uid: 0026cc97-3536-11e8-bd83-0800274106a1 -spec: - size: 3 -status: - nodes: - - memcached-sample-6fd7c98d8-7dqdr - - memcached-sample-6fd7c98d8-g5k7v - - memcached-sample-6fd7c98d8-m7vn7 +$ kubectl apply -f config/samples/cache_v1alpha1_memcached.yaml -n memcached-operator-system ``` -### Update the size +## Uninstall CRDs -Update `config/samples/cache_v1alpha1_memcached.yaml` to change the `spec.size` field in the Memcached CR from 3 to 5: +To delete your CRDs from the cluster: ```sh -$ kubectl patch memcached memcached-sample -p '{"spec":{"size": 5}}' --type=merge +$ make uninstall ``` -Confirm that the operator changes the deployment size: +## Undeploy Project -```console -$ kubectl get deployment -NAME READY UP-TO-DATE AVAILABLE AGE -memcached-operator-controller-manager 1/1 1 1 10m -memcached-sample 5/5 5 5 3m -``` +To undeploy and remove the manifests from the cluster. -### Cleanup +- Add the following target to your Makefile : ```sh -$ kubectl delete -f config/samples/cache_v1alpha1_memcached.yaml -$ kubectl delete deployments,service -l control-plane=controller-manager -$ kubectl delete role,rolebinding --all +# UnDeploy controller from the configured Kubernetes cluster in ~/.kube/config +undeploy: + $(KUSTOMIZE) build config/default | kubectl delete -f - ``` +- Run `make undeploy` -## Further steps - -The following guides build off the operator created in this example, adding advanced features: - -- [Create a validating or mutating Admission Webhook][create_a_webhook] - -Also see the [advanced topics][advanced_topics] doc for more use cases and under the hood details. +## Next Step +Now, follow up the [Tutorial][tutorial] to better understand how it works by developing a demo project. [go_tool]:https://golang.org/dl/ [docker_tool]:https://docs.docker.com/install/ [kubectl_tool]:https://kubernetes.io/docs/tasks/tools/install-kubectl/ -[kustomize_tool]: https://sigs.k8s.io/kustomize/docs/INSTALL.md - -[enqueue_requests_from_map_func]: https://godoc.org/sigs.k8s.io/controller-runtime/pkg/handler#EnqueueRequestsFromMapFunc -[event_handler_godocs]: https://godoc.org/sigs.k8s.io/controller-runtime/pkg/handler#hdr-EventHandlers -[event_filtering]:/docs/building-operators/golang/references/event-filtering/ -[controller_options]: https://godoc.org/github.com/kubernetes-sigs/controller-runtime/pkg/controller#Options -[controller_godocs]: https://godoc.org/github.com/kubernetes-sigs/controller-runtime/pkg/controller -[operator_scope]:/docs/building-operators/golang/operator-scope/ -[kubebuilder_layout_doc]:https://book.kubebuilder.io/cronjob-tutorial/basic-project.html -[homebrew_tool]:https://brew.sh/ -[go_mod_wiki]: https://github.com/golang/go/wiki/Modules -[go_vendoring]: https://blog.gopheracademy.com/advent-2015/vendor-folder/ -[doc_client_api]:/docs/building-operators/golang/references/client/ -[manager_go_doc]: https://godoc.org/github.com/kubernetes-sigs/controller-runtime/pkg/manager#Manager -[controller-go-doc]: https://godoc.org/github.com/kubernetes-sigs/controller-runtime/pkg#hdr-Controller -[request-go-doc]: https://godoc.org/github.com/kubernetes-sigs/controller-runtime/pkg/reconcile#Request -[result_go_doc]: https://godoc.org/github.com/kubernetes-sigs/controller-runtime/pkg/reconcile#Result -[multi-namespaced-cache-builder]: https://godoc.org/github.com/kubernetes-sigs/controller-runtime/pkg/cache#MultiNamespacedCacheBuilder -[cli-run-olm]: /docs/olm-integration/cli-overview -[kubebuilder_entrypoint_doc]: https://book.kubebuilder.io/cronjob-tutorial/empty-main.html - -[api_terms_doc]: https://book.kubebuilder.io/cronjob-tutorial/gvks.html -[kb_controller_doc]: https://book.kubebuilder.io/cronjob-tutorial/controller-overview.html -[kb_api_doc]: https://book.kubebuilder.io/cronjob-tutorial/new-api.html -[controller_tools]: https://sigs.k8s.io/controller-tools -[doc-validation-schema]: https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#specifying-a-structural-schema -[generating-crd]: https://book.kubebuilder.io/reference/generating-crd.html -[markers]: https://book.kubebuilder.io/reference/markers.html -[crd-markers]: https://book.kubebuilder.io/reference/markers/crd-validation.html -[rbac-markers]: https://book.kubebuilder.io/reference/markers/rbac.html -[memcached_controller]: https://github.com/operator-framework/operator-sdk/blob/master/example/memcached-operator/memcached_controller.go.tmpl -[builder_godocs]: https://godoc.org/github.com/kubernetes-sigs/controller-runtime/pkg/builder#example-Builder -[legacy_quickstart_doc]:https://v0-19-x.sdk.operatorframework.io/docs/golang/legacy/quickstart/ -[activate_modules]: https://github.com/golang/go/wiki/Modules#how-to-install-and-activate-module-support -[advanced_topics]: /docs/building-operators/golang/advanced-topics/ -[create_a_webhook]: /docs/building-operators/golang/webhooks/ -[status_marker]: https://book.kubebuilder.io/reference/generating-crd.html#status -[status_subresource]: https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#status-subresource -[API-groups]:https://kubernetes.io/docs/concepts/overview/kubernetes-api/#api-groups -[legacy_CLI]:https://v0-19-x.sdk.operatorframework.io/docs/cli/ -[env-test-setup]: /docs/building-operators/golang/references/env-test-setup \ No newline at end of file +[operator_install]: /docs/installation/install-operator-sdk +[env-test-setup]: /docs/building-operators/golang/references/env-test-setup +[tutorial]: /docs/building-operators/golang/tutorial/ diff --git a/website/content/en/docs/building-operators/golang/tutorial.md b/website/content/en/docs/building-operators/golang/tutorial.md new file mode 100644 index 0000000000..638436760a --- /dev/null +++ b/website/content/en/docs/building-operators/golang/tutorial.md @@ -0,0 +1,450 @@ +--- +title: Golang Based Operator Tutorial +linkTitle: Tutorial +weight: 30 +--- + +**NOTE:** For the SDK versions prior to `v0.19.0` please consult the [legacy docs][legacy_quickstart_doc] for the [legacy CLI][legacy_CLI] and project. + +This guide walks through an example of building a simple memcached-operator using the operator-sdk CLI tool and controller-runtime library API. + +## Prerequisites + +- [go][go_tool] version v1.13+. +- [docker][docker_tool] version 17.03+. +- [kubectl][kubectl_tool] version v1.11.3+. +- [kustomize][kustomize_tool] v3.1.0+ +- Access to a Kubernetes v1.11.3+ cluster. + +## Create a new project + +Use the CLI to create a new memcached-operator project: + +```sh +$ mkdir -p $HOME/projects/memcached-operator +$ cd $HOME/projects/memcached-operator +# we'll use a domain of example.com +# so all API groups will be .example.com +$ operator-sdk init --domain=example.com --repo=github.com/example-inc/memcached-operator +``` + +To learn about the project directory structure, see [Kubebuilder project layout][kubebuilder_layout_doc] doc. + +#### A note on dependency management + +`operator-sdk init` generates a `go.mod` file to be used with [Go modules][go_mod_wiki]. The `--repo=` flag is required when creating a project outside of `$GOPATH/src`, as scaffolded files require a valid module path. Ensure you [activate module support][activate_modules] by running `export GO111MODULE=on` before using the SDK. + +### Manager +The main program for the operator `main.go` initializes and runs the [Manager][manager_go_doc]. + +See the [Kubebuilder entrypoint doc][kubebuilder_entrypoint_doc] for more details on how the manager registers the Scheme for the custom resource API defintions, and sets up and runs controllers and webhooks. + + +The Manager can restrict the namespace that all controllers will watch for resources: +```Go +mgr, err := manager.New(cfg, manager.Options{Namespace: namespace}) +``` +By default this will be the namespace that the operator is running in. To watch all namespaces leave the namespace option empty: +```Go +mgr, err := manager.New(cfg, manager.Options{Namespace: ""}) +``` + +It is also possible to use the [MultiNamespacedCacheBuilder][multi-namespaced-cache-builder] to watch a specific set of namespaces: +```Go +var namespaces []string // List of Namespaces +// Create a new Cmd to provide shared dependencies and start components +mgr, err := manager.New(cfg, manager.Options{ + NewCache: cache.MultiNamespacedCacheBuilder(namespaces), +}) +``` + +#### Operator scope + +Read the [operator scope][operator_scope] documentation on how to run your operator as namespace-scoped vs cluster-scoped. + +By default the main program will set the manager's namespace using the value of `WATCH_NAMESPACE` env defined in `deploy/operator.yaml`. + +### Multi-Group APIs + +Before creating an API and controller, consider if your operator's API requires multiple [groups][API-groups]. +If yes then add the line `multigroup: true` in the `PROJECT` file which should look like the following: + +```YAML +domain: example.com +layout: go.kubebuilder.io/v2 +multigroup: true +... +``` +For multi-group projects, the API Go type files are created under `apis///` and the controllers under `controllers//`. + +This guide will cover the default case of a single group API. + +## Create a new API and Controller + +Create a new Custom Resource Definition(CRD) API with group `cache` version `v1alpha1` and Kind Memcached. +When prompted, enter yes `y` for creating both the resource and controller. + +```console +$ operator-sdk create api --group=cache --version=v1alpha1 --kind=Memcached +Create Resource [y/n] +y +Create Controller [y/n] +y +Writing scaffold for you to edit... +api/v1alpha1/memcached_types.go +controllers/memcached_controller.go +... +``` + +This will scaffold the Memcached resource API at `api/v1alpha1/memcached_types.go` and the controller at `controllers/memcached_controller.go`. + +See the [API terminology doc][api_terms_doc] for details on the CRD API conventions. + +To understand the API Go types and controller scaffolding see the Kubebuilder [api doc][kb_api_doc] and [controller doc][kb_controller_doc]. + +### Define the API + +Define the API for the Memcached Custom Resource(CR) by modifying the Go type definitions at `api/v1alpha1/memcached_types.go` to have the following spec and status: + +```Go +// MemcachedSpec defines the desired state of Memcached +type MemcachedSpec struct { + // +kubebuilder:validation:Minimum=0 + // Size is the size of the memcached deployment + Size int32 `json:"size"` +} + +// MemcachedStatus defines the observed state of Memcached +type MemcachedStatus struct { + // Nodes are the names of the memcached pods + Nodes []string `json:"nodes"` +} +``` + +Add the `+kubebuilder:subresource:status` [marker][status_marker] to add a [status subresource][status_subresource] to the CRD manifest so that the controller can update the CR status without changing the rest of the CR object: + +```Go +// Memcached is the Schema for the memcacheds API +// +kubebuilder:subresource:status +type Memcached struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec MemcachedSpec `json:"spec,omitempty"` + Status MemcachedStatus `json:"status,omitempty"` +} +``` + +After modifying the `*_types.go` file always run the following command to update the generated code for that resource type: + +```sh +$ make generate +``` + +The above makefile target will invoke the [controller-gen][controller_tools] utility to update the `api/v1alpha1/zz_generated.deepcopy.go` file to ensure our API's Go type definitons implement the `runtime.Object` interface that all Kind types must implement. + +### Generating CRD manifests + +Once the API is defined with spec/status fields and CRD validation markers, the CRD manifests can be generated and updated with the following command: + +```console +$ make manifests +``` + +This makefile target will invoke controller-gen to generate the CRD manifests at `config/crd/bases/cache.example.com_memcacheds.yaml`. + +#### OpenAPI validation + +OpenAPIv3 schemas are added to CRD manifests in the `spec.validation` block when the manifests are generated. This validation block allows Kubernetes to validate the properties in a Memcached Custom Resource when it is created or updated. + +Markers (annotations) are available to configure validations for your API. These markers will always have a `+kubebuilder:validation` prefix. + +Usage of markers in API code is discussed in the kubebuilder [CRD generation][generating-crd] and [marker][markers] documentation. A full list of OpenAPIv3 validation markers can be found [here][crd-markers]. + +To learn more about OpenAPI v3.0 validation schemas in CRDs, refer to the [Kubernetes Documentation][doc-validation-schema]. + +### Implement the Controller + +For this example replace the generated controller file `controllers/memcached_controller.go` with the example [`memcached_controller.go`][memcached_controller] implementation. + +The example controller executes the following reconciliation logic for each Memcached CR: +- Create a memcached Deployment if it doesn't exist +- Ensure that the Deployment size is the same as specified by the Memcached CR spec +- Update the Memcached CR status using the status writer with the names of the memcached pods + +The next two subsections explain how the controller watches resources and how the reconcile loop is triggered. Skip to the [Build](#build-and-run-the-operator) section to see how to build and run the operator. + +#### Resources watched by the Controller + +The `SetupWithManager()` function in `controllers/memcached_controller.go` specifies how the controller is built to watch a CR and other resources that are owned and managed by that controller. + +```Go +func (r *MemcachedReconciler) SetupWithManager(mgr ctrl.Manager) error { + return ctrl.NewControllerManagedBy(mgr). + For(&cachev1alpha1.Memcached{}). + Owns(&appsv1.Deployment{}). + Complete(r) +} +``` + +The `NewControllerManagedBy()` provides a controller builder that allows various controller configurations. + +`For(&cachev1alpha1.Memcached{})` specifies the Memcached type as the primary resource to watch. For each Memcached type Add/Update/Delete event the reconcile loop will be sent a reconcile `Request` (a namespace/name key) for that Memcached object. + +`Owns(&appsv1.Deployment{})` specifies the Deployments type as the secondary resource to watch. For each Deployment type Add/Update/Delete event, the event handler will map each event to a reconcile `Request` for the owner of the Deployment. Which in this case is the Memcached object for which the Deployment was created. + +#### Controller Configurations + +There are a number of other useful configurations that can be made when initialzing a controller. For more details on these configurations consult the upstream [builder][builder_godocs] and [controller][controller_godocs] godocs. + +- Set the max number of concurrent Reconciles for the controller via the [`MaxConcurrentReconciles`][controller_options] option. Defaults to 1. + ```Go + func (r *MemcachedReconciler) SetupWithManager(mgr ctrl.Manager) error { + return ctrl.NewControllerManagedBy(mgr). + For(&cachev1alpha1.Memcached{}). + Owns(&appsv1.Deployment{}). + WithOptions(controller.Options{ + MaxConcurrentReconciles: 2, + }). + Complete(r) + } + ``` +- Filter watch events using [predicates][event_filtering] +- Choose the type of [EventHandler][event_handler_godocs] to change how a watch event will translate to reconcile requests for the reconcile loop. For operator relationships that are more complex than primary and secondary resources, the [`EnqueueRequestsFromMapFunc`][enqueue_requests_from_map_func] handler can be used to transform a watch event into an arbitrary set of reconcile requests. + + +#### Reconcile loop + +Every Controller has a Reconciler object with a `Reconcile()` method that implements the reconcile loop. The reconcile loop is passed the [`Request`][request-go-doc] argument which is a Namespace/Name key used to lookup the primary resource object, Memcached, from the cache: + +```Go +import ( + ctrl "sigs.k8s.io/controller-runtime" + + cachev1alpha1 "github.com/example-inc/memcached-operator/api/v1alpha1" + ... +) + +func (r *MemcachedReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error) { + // Lookup the Memcached instance for this reconcile request + memcached := &cachev1alpha1.Memcached{} + err := r.Get(ctx, req.NamespacedName, memcached) + ... +} +``` + +Based on the return values, [`Result`][result_go_doc] and error, the `Request` may be requeued and the reconcile loop may be triggered again: + +```Go +// Reconcile successful - don't requeue +return ctrl.Result{}, nil +// Reconcile failed due to error - requeue +return ctrl.Result{}, err +// Requeue for any reason other than an error +return ctrl.Result{Requeue: true}, nil +``` + +You can set the `Result.RequeueAfter` to requeue the `Request` after a grace period as well: +```Go +import "time" + +// Reconcile for any reason other than an error after 5 seconds +return ctrl.Result{RequeueAfter: time.Second*5}, nil +``` + +**Note:** Returning `Result` with `RequeueAfter` set is how you can periodically reconcile a CR. + +For a guide on Reconcilers, Clients, and interacting with resource Events, see the [Client API doc][doc_client_api]. + +### Specify permissions and generate RBAC manifests + +The controller needs certain RBAC permissions to interact with the resources it manages. These are specified via [RBAC markers][rbac_markers] like the following: + +```Go +// +kubebuilder:rbac:groups=cache.example.com,resources=memcacheds,verbs=get;list;watch;create;update;patch;delete +// +kubebuilder:rbac:groups=cache.example.com,resources=memcacheds/status,verbs=get;update;patch +// +kubebuilder:rbac:groups=apps,resources=deployments,verbs=get;list;watch;create;update;patch;delete +// +kubebuilder:rbac:groups=core,resources=pods,verbs=get;list; + +func (r *MemcachedReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error) { +``` + +The `ClusterRole` manifest at `config/rbac/role.yaml` is updated from the above [RBAC markers][rbac_markers] via controller-gen with the following command: + +```sh +$ make manifests +``` + +## Build and run the operator + +See the [quickstart guide][quickstart] to know how to build the project and run it. Also, +ensure that you have your test environment configured before run the targets to build the project. + +## Customize your Operator image + +Before building the operator image, ensure the generated Dockerfile references +the base image you want. You can change the default "runner" image `gcr.io/distroless/static:nonroot` +by replacing its tag with another, for example `alpine:latest`, and removing +the `USER: nonroot:nonroot` directive for example. + +**Note**: +The name and tag of the image (`IMG=/:tag`) in both the commands can also be set in the Makefile. Modify the line which has `IMG ?= controller:latest` to set your desired default image name. + +#### Deploy the operator + +For this example we will run the operator in the `default` namespace which can be specified for all resources in `config/default/kustomization.yaml`: + +```sh +$ cd config/default/ && kustomize edit set namespace "default" && cd ../.. +``` + +Run the following to deploy the operator. This will also install the RBAC manifests from `config/rbac`. + +```sh +$ make deploy IMG=quay.io/$USERNAME/memcached-operator:v0.0.1 +``` + +*NOTE* If you have enabled webhooks in your deployments, you will need to have cert-manager already installed +in the cluster or `make deploy` will fail when creating the cert-manager resources. + +Verify that the memcached-operator is up and running: + +```console +$ kubectl get deployment +NAME READY UP-TO-DATE AVAILABLE AGE +memcached-operator-controller-manager 1/1 1 1 8m +``` + +### 3. Deploy your Operator with the Operator Lifecycle Manager (OLM) + +OLM will manage creation of most if not all resources required to run your operator, using a bit of setup from other `operator-sdk` commands. Check out the [docs][cli-run-olm] for more information. + +## Create a Memcached CR + +Update the sample Memcached CR manifest at `config/samples/cache_v1alpha1_memcached.yaml` and define the `spec` as the following: + +```YAML +apiVersion: cache.example.com/v1alpha1 +kind: Memcached +metadata: + name: memcached-sample +spec: + size: 3 +``` + +Create the CR: + +```sh +$ kubectl apply -f config/samples/cache_v1alpha1_memcached.yaml +``` + +Ensure that the memcached operator creates the deployment for the sample CR with the correct size: + +```console +$ kubectl get deployment +NAME READY UP-TO-DATE AVAILABLE AGE +memcached-operator-controller-manager 1/1 1 1 8m +memcached-sample 3/3 3 3 1m +``` + +Check the pods and CR status to confirm the status is updated with the memcached pod names: + +```console +$ kubectl get pods +NAME READY STATUS RESTARTS AGE +memcached-sample-6fd7c98d8-7dqdr 1/1 Running 0 1m +memcached-sample-6fd7c98d8-g5k7v 1/1 Running 0 1m +memcached-sample-6fd7c98d8-m7vn7 1/1 Running 0 1m +``` + +```console +$ kubectl get memcached/memcached-sample -o yaml +apiVersion: cache.example.com/v1alpha1 +kind: Memcached +metadata: + clusterName: "" + creationTimestamp: 2018-03-31T22:51:08Z + generation: 0 + name: memcached-sample + namespace: default + resourceVersion: "245453" + selfLink: /apis/cache.example.com/v1alpha1/namespaces/default/memcacheds/memcached-sample + uid: 0026cc97-3536-11e8-bd83-0800274106a1 +spec: + size: 3 +status: + nodes: + - memcached-sample-6fd7c98d8-7dqdr + - memcached-sample-6fd7c98d8-g5k7v + - memcached-sample-6fd7c98d8-m7vn7 +``` + +### Update the size + +Update `config/samples/cache_v1alpha1_memcached.yaml` to change the `spec.size` field in the Memcached CR from 3 to 5: + +```sh +$ kubectl patch memcached memcached-sample -p '{"spec":{"size": 5}}' --type=merge +``` + +Confirm that the operator changes the deployment size: + +```console +$ kubectl get deployment +NAME READY UP-TO-DATE AVAILABLE AGE +memcached-operator-controller-manager 1/1 1 1 10m +memcached-sample 5/5 5 5 3m +``` + +## Next steps + +The following guides build off the operator created in this example, adding advanced features: + +- [Create a validating or mutating Admission Webhook][create_a_webhook] + +Also see the [advanced topics][advanced_topics] doc for more use cases and under the hood details. + +[go_tool]:https://golang.org/dl/ +[docker_tool]:https://docs.docker.com/install/ +[kubectl_tool]:https://kubernetes.io/docs/tasks/tools/install-kubectl/ +[kustomize_tool]: https://sigs.k8s.io/kustomize/docs/INSTALL.md + +[enqueue_requests_from_map_func]: https://godoc.org/sigs.k8s.io/controller-runtime/pkg/handler#EnqueueRequestsFromMapFunc +[event_handler_godocs]: https://godoc.org/sigs.k8s.io/controller-runtime/pkg/handler#hdr-EventHandlers +[event_filtering]:/docs/building-operators/golang/references/event-filtering/ +[controller_options]: https://godoc.org/github.com/kubernetes-sigs/controller-runtime/pkg/controller#Options +[controller_godocs]: https://godoc.org/github.com/kubernetes-sigs/controller-runtime/pkg/controller +[operator_scope]:/docs/building-operators/golang/operator-scope/ +[kubebuilder_layout_doc]:https://book.kubebuilder.io/cronjob-tutorial/basic-project.html +[homebrew_tool]:https://brew.sh/ +[go_mod_wiki]: https://github.com/golang/go/wiki/Modules +[go_vendoring]: https://blog.gopheracademy.com/advent-2015/vendor-folder/ +[doc_client_api]:/docs/building-operators/golang/references/client/ +[manager_go_doc]: https://godoc.org/github.com/kubernetes-sigs/controller-runtime/pkg/manager#Manager +[controller-go-doc]: https://godoc.org/github.com/kubernetes-sigs/controller-runtime/pkg#hdr-Controller +[request-go-doc]: https://godoc.org/github.com/kubernetes-sigs/controller-runtime/pkg/reconcile#Request +[result_go_doc]: https://godoc.org/github.com/kubernetes-sigs/controller-runtime/pkg/reconcile#Result +[multi-namespaced-cache-builder]: https://godoc.org/github.com/kubernetes-sigs/controller-runtime/pkg/cache#MultiNamespacedCacheBuilder +[cli-run-olm]: /docs/olm-integration/cli-overview +[kubebuilder_entrypoint_doc]: https://book.kubebuilder.io/cronjob-tutorial/empty-main.html + +[api_terms_doc]: https://book.kubebuilder.io/cronjob-tutorial/gvks.html +[kb_controller_doc]: https://book.kubebuilder.io/cronjob-tutorial/controller-overview.html +[kb_api_doc]: https://book.kubebuilder.io/cronjob-tutorial/new-api.html +[controller_tools]: https://sigs.k8s.io/controller-tools +[doc-validation-schema]: https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#specifying-a-structural-schema +[generating-crd]: https://book.kubebuilder.io/reference/generating-crd.html +[markers]: https://book.kubebuilder.io/reference/markers.html +[crd-markers]: https://book.kubebuilder.io/reference/markers/crd-validation.html +[rbac-markers]: https://book.kubebuilder.io/reference/markers/rbac.html +[memcached_controller]: https://github.com/operator-framework/operator-sdk/blob/master/example/memcached-operator/memcached_controller.go.tmpl +[builder_godocs]: https://godoc.org/github.com/kubernetes-sigs/controller-runtime/pkg/builder#example-Builder +[legacy_quickstart_doc]:https://v0-19-x.sdk.operatorframework.io/docs/golang/legacy/quickstart/ +[activate_modules]: https://github.com/golang/go/wiki/Modules#how-to-install-and-activate-module-support +[advanced_topics]: /docs/building-operators/golang/advanced-topics/ +[create_a_webhook]: /docs/building-operators/golang/webhooks/ +[status_marker]: https://book.kubebuilder.io/reference/generating-crd.html#status +[status_subresource]: https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#status-subresource +[API-groups]:https://kubernetes.io/docs/concepts/overview/kubernetes-api/#api-groups +[legacy_CLI]:https://v0-19-x.sdk.operatorframework.io/docs/cli/ +[env-test-setup]: /docs/building-operators/golang/references/env-test-setup \ No newline at end of file diff --git a/website/content/en/docs/building-operators/golang/webhooks.md b/website/content/en/docs/building-operators/golang/webhooks.md index f5e327ccde..0717efebfd 100644 --- a/website/content/en/docs/building-operators/golang/webhooks.md +++ b/website/content/en/docs/building-operators/golang/webhooks.md @@ -115,15 +115,14 @@ If your certificates are properly configured, you should be able to start your o $ make run ENABLE_WEBHOOKS=true ``` -### Run as a Deployment inside the cluster - -For instructions on deploying your operator into a cluster, refer to the [quickstart guide][quickstart_run_as_deployment] instructions. -Adding webhooks does not alter this step. +## Build and run the operator +See the [quickstart guide][quickstart] to know how to build the project and run it. Also, +ensure that you have your test environment configured before run the targets to build the project. ## Create a Memcached CR to exercise your webhook -First, follow the instructions for creating your Memcached CR in the [quickstart guide][quickstart_create_a_cr]. +First, follow the instructions for creating your Memcached CR in the [tutorial_guide][tutorial_create_a_cr]. Once you have completed this step, you should have a Memcached CR with a size of 3 and a Memcached deployment with 3 replicas in your cluster. @@ -201,8 +200,8 @@ memcached-operator-controller-manager 1/1 1 1 10m memcached-sample 5/5 5 5 3m ``` -[quickstart_run_as_deployment]: /docs/building-operators/golang/quickstart/#2-run-as-a-deployment-inside-the-cluster -[quickstart_create_a_cr]: /docs/building-operators/golang/quickstart/#create-a-memcached-cr +[quickstart]: /docs/building-operators/golang/quickstart/ +[tutorial_create_a_cr]: /docs/building-operators/golang/tutorial/#create-a-memcached-cr [kubebuilder_admission_controllers]: https://book.kubebuilder.io/reference/admission-webhook.html [kubebuilder_cronjob_webhook]: https://book.kubebuilder.io/cronjob-tutorial/webhook-implementation.html From 04c48908ca7b675097c310068852fe9d65be96f8 Mon Sep 17 00:00:00 2001 From: Camila Macedo Date: Tue, 28 Jul 2020 10:10:55 +0100 Subject: [PATCH 2/3] doc: organize tutorial to centralize webhooks info in its doc --- .../building-operators/golang/tutorial.md | 101 ++++++------------ .../building-operators/golang/webhooks.md | 21 ++-- 2 files changed, 48 insertions(+), 74 deletions(-) diff --git a/website/content/en/docs/building-operators/golang/tutorial.md b/website/content/en/docs/building-operators/golang/tutorial.md index 638436760a..3cafd85a79 100644 --- a/website/content/en/docs/building-operators/golang/tutorial.md +++ b/website/content/en/docs/building-operators/golang/tutorial.md @@ -8,63 +8,36 @@ weight: 30 This guide walks through an example of building a simple memcached-operator using the operator-sdk CLI tool and controller-runtime library API. -## Prerequisites +## Creating a new project -- [go][go_tool] version v1.13+. -- [docker][docker_tool] version 17.03+. -- [kubectl][kubectl_tool] version v1.11.3+. -- [kustomize][kustomize_tool] v3.1.0+ -- Access to a Kubernetes v1.11.3+ cluster. +In Kubebuilder-style projects, CRD groups are defined using two different flags +(`--group` and `--domain`). -## Create a new project +When we initialize a new project, we need to specify the domain that _all_ APIs in +our project will share, so before creating the new project, we can to determine which +domain we're using for the APIs in our existing project. -Use the CLI to create a new memcached-operator project: +The domain is everything after the first DNS segment. Using `cache.example.com` as an +example, the `--domain` would be `example.com`. +In this example, we also will use the flag `--repo` which will set the path for its go module. Note that `operator-sdk init` generates a `go.mod` file to be used with [Go modules][go_mod_wiki]. The `--repo=` flag is required when creating a project outside of `$GOPATH/src`, as scaffolded files require a valid module path. Ensure you [activate module support][activate_modules] by running `export GO111MODULE=on` before using the SDK. + + ```sh -$ mkdir -p $HOME/projects/memcached-operator -$ cd $HOME/projects/memcached-operator -# we'll use a domain of example.com -# so all API groups will be .example.com +$ mkdir $GOPATH/src/memcached-operator +$ cd $GOPATH/src/memcached-operator $ operator-sdk init --domain=example.com --repo=github.com/example-inc/memcached-operator ``` -To learn about the project directory structure, see [Kubebuilder project layout][kubebuilder_layout_doc] doc. - -#### A note on dependency management - -`operator-sdk init` generates a `go.mod` file to be used with [Go modules][go_mod_wiki]. The `--repo=` flag is required when creating a project outside of `$GOPATH/src`, as scaffolded files require a valid module path. Ensure you [activate module support][activate_modules] by running `export GO111MODULE=on` before using the SDK. +**Note** If you’re not in GOPATH, you’ll need to run `go mod init ` in order to tell kubebuilder and Go the base import path of your module. For further information see the [How to Write Go Code][godoc] into the Golang doc page. -### Manager +## Customizing the initialization + The main program for the operator `main.go` initializes and runs the [Manager][manager_go_doc]. -See the [Kubebuilder entrypoint doc][kubebuilder_entrypoint_doc] for more details on how the manager registers the Scheme for the custom resource API defintions, and sets up and runs controllers and webhooks. +See the [Kubebuilder entrypoint doc][kubebuilder_entrypoint_doc] for more details over the options to instantiate a manager. - -The Manager can restrict the namespace that all controllers will watch for resources: -```Go -mgr, err := manager.New(cfg, manager.Options{Namespace: namespace}) -``` -By default this will be the namespace that the operator is running in. To watch all namespaces leave the namespace option empty: -```Go -mgr, err := manager.New(cfg, manager.Options{Namespace: ""}) -``` - -It is also possible to use the [MultiNamespacedCacheBuilder][multi-namespaced-cache-builder] to watch a specific set of namespaces: -```Go -var namespaces []string // List of Namespaces -// Create a new Cmd to provide shared dependencies and start components -mgr, err := manager.New(cfg, manager.Options{ - NewCache: cache.MultiNamespacedCacheBuilder(namespaces), -}) -``` - -#### Operator scope - -Read the [operator scope][operator_scope] documentation on how to run your operator as namespace-scoped vs cluster-scoped. - -By default the main program will set the manager's namespace using the value of `WATCH_NAMESPACE` env defined in `deploy/operator.yaml`. - -### Multi-Group APIs +## Multi-Group APIs Before creating an API and controller, consider if your operator's API requires multiple [groups][API-groups]. If yes then add the line `multigroup: true` in the `PROJECT` file which should look like the following: @@ -79,7 +52,7 @@ For multi-group projects, the API Go type files are created under `apis// This guide will cover the default case of a single group API. -## Create a new API and Controller +## Creating a new API and Controller Create a new Custom Resource Definition(CRD) API with group `cache` version `v1alpha1` and Kind Memcached. When prompted, enter yes `y` for creating both the resource and controller. @@ -102,7 +75,7 @@ See the [API terminology doc][api_terms_doc] for details on the CRD API conventi To understand the API Go types and controller scaffolding see the Kubebuilder [api doc][kb_api_doc] and [controller doc][kb_controller_doc]. -### Define the API +## Defining the API Define the API for the Memcached Custom Resource(CR) by modifying the Go type definitions at `api/v1alpha1/memcached_types.go` to have the following spec and status: @@ -143,7 +116,7 @@ $ make generate The above makefile target will invoke the [controller-gen][controller_tools] utility to update the `api/v1alpha1/zz_generated.deepcopy.go` file to ensure our API's Go type definitons implement the `runtime.Object` interface that all Kind types must implement. -### Generating CRD manifests +## Generating CRD manifests Once the API is defined with spec/status fields and CRD validation markers, the CRD manifests can be generated and updated with the following command: @@ -153,7 +126,7 @@ $ make manifests This makefile target will invoke controller-gen to generate the CRD manifests at `config/crd/bases/cache.example.com_memcacheds.yaml`. -#### OpenAPI validation +### OpenAPI validation OpenAPIv3 schemas are added to CRD manifests in the `spec.validation` block when the manifests are generated. This validation block allows Kubernetes to validate the properties in a Memcached Custom Resource when it is created or updated. @@ -163,7 +136,7 @@ Usage of markers in API code is discussed in the kubebuilder [CRD generation][ge To learn more about OpenAPI v3.0 validation schemas in CRDs, refer to the [Kubernetes Documentation][doc-validation-schema]. -### Implement the Controller +## Implementing the Controller For this example replace the generated controller file `controllers/memcached_controller.go` with the example [`memcached_controller.go`][memcached_controller] implementation. @@ -174,7 +147,7 @@ The example controller executes the following reconciliation logic for each Memc The next two subsections explain how the controller watches resources and how the reconcile loop is triggered. Skip to the [Build](#build-and-run-the-operator) section to see how to build and run the operator. -#### Resources watched by the Controller +### Resources watched by the Controller The `SetupWithManager()` function in `controllers/memcached_controller.go` specifies how the controller is built to watch a CR and other resources that are owned and managed by that controller. @@ -193,7 +166,7 @@ The `NewControllerManagedBy()` provides a controller builder that allows various `Owns(&appsv1.Deployment{})` specifies the Deployments type as the secondary resource to watch. For each Deployment type Add/Update/Delete event, the event handler will map each event to a reconcile `Request` for the owner of the Deployment. Which in this case is the Memcached object for which the Deployment was created. -#### Controller Configurations +### Controller Configurations There are a number of other useful configurations that can be made when initialzing a controller. For more details on these configurations consult the upstream [builder][builder_godocs] and [controller][controller_godocs] godocs. @@ -213,7 +186,7 @@ There are a number of other useful configurations that can be made when initialz - Choose the type of [EventHandler][event_handler_godocs] to change how a watch event will translate to reconcile requests for the reconcile loop. For operator relationships that are more complex than primary and secondary resources, the [`EnqueueRequestsFromMapFunc`][enqueue_requests_from_map_func] handler can be used to transform a watch event into an arbitrary set of reconcile requests. -#### Reconcile loop +### Reconcile loop Every Controller has a Reconciler object with a `Reconcile()` method that implements the reconcile loop. The reconcile loop is passed the [`Request`][request-go-doc] argument which is a Namespace/Name key used to lookup the primary resource object, Memcached, from the cache: @@ -275,12 +248,14 @@ The `ClusterRole` manifest at `config/rbac/role.yaml` is updated from the above $ make manifests ``` -## Build and run the operator +By default, the projects are `cluster-scoped`. Check that your role is of the `ClusterRole` Kind. For further information check the [operator-scope][operator-scope] doc. + +## Building and running the operator See the [quickstart guide][quickstart] to know how to build the project and run it. Also, ensure that you have your test environment configured before run the targets to build the project. -## Customize your Operator image +## Customizing your Operator image Before building the operator image, ensure the generated Dockerfile references the base image you want. You can change the default "runner" image `gcr.io/distroless/static:nonroot` @@ -290,7 +265,7 @@ the `USER: nonroot:nonroot` directive for example. **Note**: The name and tag of the image (`IMG=/:tag`) in both the commands can also be set in the Makefile. Modify the line which has `IMG ?= controller:latest` to set your desired default image name. -#### Deploy the operator +## Deploying the Operator For this example we will run the operator in the `default` namespace which can be specified for all resources in `config/default/kustomization.yaml`: @@ -304,9 +279,6 @@ Run the following to deploy the operator. This will also install the RBAC manife $ make deploy IMG=quay.io/$USERNAME/memcached-operator:v0.0.1 ``` -*NOTE* If you have enabled webhooks in your deployments, you will need to have cert-manager already installed -in the cluster or `make deploy` will fail when creating the cert-manager resources. - Verify that the memcached-operator is up and running: ```console @@ -315,11 +287,7 @@ NAME READY UP-TO-DATE AVAILABLE AGE memcached-operator-controller-manager 1/1 1 1 8m ``` -### 3. Deploy your Operator with the Operator Lifecycle Manager (OLM) - -OLM will manage creation of most if not all resources required to run your operator, using a bit of setup from other `operator-sdk` commands. Check out the [docs][cli-run-olm] for more information. - -## Create a Memcached CR +## Creating an instance of a Memcached CR Update the sample Memcached CR manifest at `config/samples/cache_v1alpha1_memcached.yaml` and define the `spec` as the following: @@ -379,7 +347,7 @@ status: - memcached-sample-6fd7c98d8-m7vn7 ``` -### Update the size +## Updating the size of the Memcached instances Update `config/samples/cache_v1alpha1_memcached.yaml` to change the `spec.size` field in the Memcached CR from 3 to 5: @@ -447,4 +415,5 @@ Also see the [advanced topics][advanced_topics] doc for more use cases and under [status_subresource]: https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#status-subresource [API-groups]:https://kubernetes.io/docs/concepts/overview/kubernetes-api/#api-groups [legacy_CLI]:https://v0-19-x.sdk.operatorframework.io/docs/cli/ -[env-test-setup]: /docs/building-operators/golang/references/env-test-setup \ No newline at end of file +[env-test-setup]: /docs/building-operators/golang/references/env-test-setup +[godoc]:https://golang.org/doc/code.html \ No newline at end of file diff --git a/website/content/en/docs/building-operators/golang/webhooks.md b/website/content/en/docs/building-operators/golang/webhooks.md index 0717efebfd..76355977e7 100644 --- a/website/content/en/docs/building-operators/golang/webhooks.md +++ b/website/content/en/docs/building-operators/golang/webhooks.md @@ -4,7 +4,7 @@ linkTitle: Admission Webhooks weight: 40 --- -## Create a validating or mutating Admission Webhook +## Creating a validating or mutating Admission Webhook An admission webhook is an HTTP callback that is registered with Kubernetes, and will be called by Kubernetes to validate or mutate a resource before being stored. There are two types of admission webhooks, validating and mutating. Validating webhooks can be used @@ -73,7 +73,7 @@ func (r *Memcached) ValidateUpdate(old runtime.Object) error { } ``` -### Generate webhook manifests and enable webhook deployment +## Generating webhook manifests Once your webhooks are implemented, all that's left is to create the `WebhookConfiguration` manifests required to register your webhooks with Kubernetes: @@ -86,7 +86,7 @@ You will need to enable cert-manager and webhook deployment in order to deploy t To do so, edit the `config/default/kustomize.yaml` and uncomment the sections marked by `[WEBHOOK]` and `[CERTMANAGER]` comments. More detail on this step can be found in the [Kubebuilder documentation][kubebuilder_running_webhook]. -### Update main.go so that running locally works +## Updating main.go so that running locally works To ensure that running locally continues working, ensure that there is a check that prevents the webhooks from being started when the `ENABLE_WEBHOOKS` flag is set to false. To do so, @@ -101,7 +101,11 @@ if os.Getenv("ENABLE_WEBHOOKS") != "false" { } ``` -## Run your operator and webhooks +## Running your operator with Webhooks + +### Configuring Cert-Manager + +Please follow [this][certmanager] guide to install cert-mamager into cluster prior to deployment. ### Run locally @@ -115,12 +119,12 @@ If your certificates are properly configured, you should be able to start your o $ make run ENABLE_WEBHOOKS=true ``` -## Build and run the operator +### Building and runnning the operator See the [quickstart guide][quickstart] to know how to build the project and run it. Also, ensure that you have your test environment configured before run the targets to build the project. -## Create a Memcached CR to exercise your webhook +### Creating a Memcached CR to exercise your webhook First, follow the instructions for creating your Memcached CR in the [tutorial_guide][tutorial_create_a_cr]. @@ -159,7 +163,7 @@ memcached-operator-controller-manager 1/1 1 1 8m memcached-sample 3/3 3 3 1m ``` -### Update the size +### Updating the size Update `config/samples/cache_v1alpha1_memcached.yaml` to change the `spec.size` field in the Memcached CR from 3 to 5: @@ -176,7 +180,7 @@ memcached-operator-controller-manager 1/1 1 1 10m memcached-sample 5/5 5 5 3m ``` -#### Update the size to an even number +### Updating the size to an even number Update `config/samples/cache_v1alpha1_memcached.yaml` to change the `spec.size` field in the Memcached CR from 3 to 4: @@ -207,3 +211,4 @@ memcached-sample 5/5 5 5 3m [kubebuilder_cronjob_webhook]: https://book.kubebuilder.io/cronjob-tutorial/webhook-implementation.html [kubebuilder_running_webhook]: https://book.kubebuilder.io/cronjob-tutorial/running-webhook.html [kubernetes_admission_controllers]: https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/ +[certmanager]: https://cert-manager.io/docs/installation/kubernetes/ \ No newline at end of file From 3b3a7a630d06a046b149c3a7dd64d13e860ad24c Mon Sep 17 00:00:00 2001 From: Camila Macedo Date: Tue, 28 Jul 2020 10:40:00 +0100 Subject: [PATCH 3/3] fix links --- .../content/en/docs/building-operators/golang/crds-scope.md | 5 ++--- .../content/en/docs/building-operators/golang/tutorial.md | 2 +- .../content/en/docs/building-operators/golang/webhooks.md | 2 +- 3 files changed, 4 insertions(+), 5 deletions(-) diff --git a/website/content/en/docs/building-operators/golang/crds-scope.md b/website/content/en/docs/building-operators/golang/crds-scope.md index 6e85b58167..365d57d1ab 100644 --- a/website/content/en/docs/building-operators/golang/crds-scope.md +++ b/website/content/en/docs/building-operators/golang/crds-scope.md @@ -26,8 +26,7 @@ one namespace in a cluster. **NOTE**: When a `Manager` instance is created in the `main.go` file, it receives the namespace(s) as Options. These namespace(s) should be watched and cached for the Client which is provided by the Controllers. Only clients provided by cluster-scoped projects where the `Namespace` attribute is `""` will be able to manage cluster-scoped CRD's. -For more information see the [Manager][manager_user_guide] topic in the user guide and the -[Manager Options][manager_options]. +For more information over the [Manager][manager_go_doc] see the [Kubebuilder entrypoint doc][kubebuilder_entrypoint_doc]. ## Example for changing the CRD scope from Namespaced to Cluster @@ -97,5 +96,5 @@ spec: ``` [RBAC]: https://kubernetes.io/docs/reference/access-authn-authz/rbac/ -[manager_user_guide]:/docs/building-operators/golang/tutorial/#manager +[kubebuilder_entrypoint_doc]:https://book.kubebuilder.io/cronjob-tutorial/empty-main.html [manager_options]: https://godoc.org/github.com/kubernetes-sigs/controller-runtime/pkg/manager#Options diff --git a/website/content/en/docs/building-operators/golang/tutorial.md b/website/content/en/docs/building-operators/golang/tutorial.md index 3cafd85a79..aecb51d160 100644 --- a/website/content/en/docs/building-operators/golang/tutorial.md +++ b/website/content/en/docs/building-operators/golang/tutorial.md @@ -145,7 +145,7 @@ The example controller executes the following reconciliation logic for each Memc - Ensure that the Deployment size is the same as specified by the Memcached CR spec - Update the Memcached CR status using the status writer with the names of the memcached pods -The next two subsections explain how the controller watches resources and how the reconcile loop is triggered. Skip to the [Build](#build-and-run-the-operator) section to see how to build and run the operator. +The next two subsections explain how the controller watches resources and how the reconcile loop is triggered. ### Resources watched by the Controller diff --git a/website/content/en/docs/building-operators/golang/webhooks.md b/website/content/en/docs/building-operators/golang/webhooks.md index 76355977e7..3b726aa24b 100644 --- a/website/content/en/docs/building-operators/golang/webhooks.md +++ b/website/content/en/docs/building-operators/golang/webhooks.md @@ -205,7 +205,7 @@ memcached-sample 5/5 5 5 3m ``` [quickstart]: /docs/building-operators/golang/quickstart/ -[tutorial_create_a_cr]: /docs/building-operators/golang/tutorial/#create-a-memcached-cr +[tutorial_create_a_cr]: /docs/building-operators/golang/tutorial/#creating-a-new-api-and-controller [kubebuilder_admission_controllers]: https://book.kubebuilder.io/reference/admission-webhook.html [kubebuilder_cronjob_webhook]: https://book.kubebuilder.io/cronjob-tutorial/webhook-implementation.html