From 2c1f2886f06114b98b50d0c671f498b82ae59eb6 Mon Sep 17 00:00:00 2001 From: Dylan Murray Date: Tue, 2 Oct 2018 10:20:35 -0400 Subject: [PATCH 01/14] Add Ansible Operator documentation --- doc/ansible/project_layout.md | 19 +++ doc/ansible/user-guide.md | 304 ++++++++++++++++++++++++++++++++++ 2 files changed, 323 insertions(+) create mode 100644 doc/ansible/project_layout.md create mode 100644 doc/ansible/user-guide.md diff --git a/doc/ansible/project_layout.md b/doc/ansible/project_layout.md new file mode 100644 index 0000000000..031f1c8274 --- /dev/null +++ b/doc/ansible/project_layout.md @@ -0,0 +1,19 @@ +# Project Scaffolding Layout + +After creating a new operator project using +`operator-sdk new`, the project directory has numerous generated folders and files. The following table describes a basic rundown of each generated file/directory. + + +| File/Folders | Purpose | +| :--- | :--- | +| Gopkg.toml Gopkg.lock | The [Go Dep][dep] manifests that describe the external dependencies of this operator. | +| cmd | Contains `main.go` which is the entry point to initialize and start this operator using the operator-sdk APIs. | +| config | Contains metadata about state of this project such as project name, kind, api-version, and so forth. The operator-sdk commands use this metadata to perform actions that require knowing the state. | +| deploy | Contains a generic set of kubernetes manifests for deploying this operator on a kubernetes cluster. | +| pkg/apis | Contains the directory tree that defines the APIs and types of Custom Resource Definitions(CRD). These files allow the sdk to do code generation for CRD types and register the schemes for all types in order to correctly decode Custom Resource objects. | +| pkg/stub | Contains `handler.go` which is the place for a user to write all the operating business logic. | +| tmp | Contains scripts that the operator-sdk uses for build and code generation. | +| vendor | The golang [vendor][Vendor] folder that contains the local copies of the external dependencies that satisfy the imports of this project. [Go Dep][dep] manages the vendor directly. | + +[Vendor]: https://golang.org/cmd/go/#hdr-Vendor_Directories +[dep]: https://github.com/golang/dep diff --git a/doc/ansible/user-guide.md b/doc/ansible/user-guide.md new file mode 100644 index 0000000000..e89221a00a --- /dev/null +++ b/doc/ansible/user-guide.md @@ -0,0 +1,304 @@ +# User Guide + +This guide walks through an example of building a simple memcached-operator powered by Ansible using tools and libraries provided by the Operator SDK. + +## Prerequisites + +- [dep][dep_tool] version v0.5.0+. +- [git][git_tool] +- [go][go_tool] version v1.10+. +- [docker][docker_tool] version 17.03+. +- [kubectl][kubectl_tool] version v1.9.0+. +- [ansible][ansible_tool] version v2.6.0+ +- Access to a kubernetes v.1.9.0+ cluster. + +**Note**: This guide uses [minikube][minikube_tool] version v0.25.0+ as the local kubernetes cluster and quay.io for the public registry. + +## Install the Operator SDK CLI + +The Operator SDK has a CLI tool that helps the developer to create, build, and deploy a new operator project. + +Checkout the desired release tag and install the SDK CLI tool: + +```sh +$ mkdir -p $GOPATH/src/github.com/operator-framework +$ cd $GOPATH/src/github.com/operator-framework +$ git clone https://github.com/operator-framework/operator-sdk +$ cd operator-sdk +$ git checkout master +$ make dep +$ make install +``` + +This installs the CLI binary `operator-sdk` at `$GOPATH/bin`. + +## Create a new project + +Use the CLI to create a new Ansible-based memcached-operator project: + +```sh +$ mkdir -p $GOPATH/src/github.com/example-inc/ +$ cd $GOPATH/src/github.com/example-inc/ +$ operator-sdk new memcached-operator --api-version=cache.example.com/v1alpha1 --kind=Memcached --type=ansible +$ cd memcached-operator +``` + +This creates the memcached-operator project specifically for watching the Memcached resource with APIVersion `cache.example.com/v1apha1` and Kind `Memcached`. + +To learn more about the project directory structure, see [project layout][layout_doc] doc. + +## Customize the operator logic + +For this example the memcached-operator will execute the following reconciliation logic for each `Memcached` CR: +- Create a memcached Deployment if it doesn't exist +- Ensure that the Deployment size is the same as specified by the `Memcached` CR spec + +### Watch the Memcached CR + +By default, the memcached-operator watches `Memcached` resource events as shown in `watches.yaml` and executes Ansible Role `Memached`: + +```yaml +--- +- version: v1alpha1 + group: cache.example.com + kind: Memcached +``` + +#### Options +**Role** +Specifying a `role` option in `watches.yaml` will configure the operator to use this specified path when launching `ansible-runner` with an Ansible Role. This is the default. +```yaml +--- +- version: v1alpha1 + group: cache.example.com + kind: Memcached + role: /opt/ansible/roles/Memcached +``` + +**Playbook** +Specifying a `playbook` option in `watches.yaml` will configure the operator to use this specified path when launching `ansible-runner` with an Ansible Playbook +```yaml +--- +- version: v1alpha1 + group: cache.example.com + kind: Memcached + playbook: /opt/ansible/playbook.yaml +``` + +## Building the Memcached Ansible Role +### Define the Memcached spec + +Defining the spec for an Ansible Operator can be done entirely in Ansible. The Ansible Operator will simply pass all key value pairs listed in the Custom Resource spec field along to Ansible as [variables](https://docs.ansible.com/ansible/2.5/user_guide/playbooks_variables.html#passing-variables-on-the-command-line). It is recommended that you perform some type validation in Ansible on the variables to ensure that your application is receiving expected input. + +First, set a default in case the user doesn't set the `spec` field by modifying `roles/Memcached/defaults/main.yml`: +```yaml +size: 1 +``` + +### Build and run the operator + +Before running the operator, Kubernetes needs to know about the new custom resource definition the operator will be watching. + +Deploy the CRD: + +```sh +$ kubectl create -f deploy/crd.yaml +``` + +Once this is done, there are two ways to run the operator: + +- As pod inside Kubernetes cluster +- As go program outside cluster using `operator-sdk` + +#### 1. Run as pod inside a Kubernetes cluster + +Run as pod inside a Kubernetes cluster is preferred for production use. + +Build the memcached-operator image and push it to a registry: +``` +$ operator-sdk build quay.io/example/memcached-operator:v0.0.1 +$ sed -i 's|REPLACE_IMAGE|quay.io/example/memcached-operator:v0.0.1|g' deploy/operator.yaml +$ docker push quay.io/example/memcached-operator:v0.0.1 +``` + +Kubernetes deployment manifests are generated in `deploy/operator.yaml`. The deployment image is set to the container image specified above. + +Deploy the memcached-operator: + +```sh +$ kubectl create -f deploy/rbac.yaml +$ kubectl create -f deploy/operator.yaml +``` + +Verify that the memcached-operator is up and running: + +```sh +$ kubectl get deployment +NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE +memcached-operator 1 1 1 1 1m +``` + +#### 2. Run outside the cluster + +This method is preferred during development cycle to deploy and test faster. + +Run the operator locally with the default kubernetes config file present at `$HOME/.kube/config`: + +```sh +$ operator-sdk up local +INFO[0000] Go Version: go1.10 +INFO[0000] Go OS/Arch: darwin/amd64 +INFO[0000] operator-sdk Version: 0.0.5+git +``` + +Run the operator locally with a provided kubernetes config file: + +```sh +$ operator-sdk up local --kubeconfig=config +INFO[0000] Go Version: go1.10 +INFO[0000] Go OS/Arch: darwin/amd64 +INFO[0000] operator-sdk Version: 0.0.5+git +``` + +### Create a Memcached CR + +Modify `deploy/cr.yaml` as shown and create a `Memcached` custom resource: + +```sh +$ cat deploy/cr.yaml +apiVersion: "cache.example.com/v1alpha1" +kind: "Memcached" +metadata: + name: "example-memcached" +spec: + size: 3 + +$ kubectl apply -f deploy/cr.yaml +``` + +Ensure that the memcached-operator creates the deployment for the CR: + +```sh +$ kubectl get deployment +NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE +memcached-operator 1 1 1 1 2m +example-memcached 3 3 3 3 1m +``` + +Check the pods and CR status to confirm the status is updated with the memcached pod names: + +```sh +$ kubectl get pods +NAME READY STATUS RESTARTS AGE +example-memcached-6fd7c98d8-7dqdr 1/1 Running 0 1m +example-memcached-6fd7c98d8-g5k7v 1/1 Running 0 1m +example-memcached-6fd7c98d8-m7vn7 1/1 Running 0 1m +memcached-operator-7cc7cfdf86-vvjqk 1/1 Running 0 2m +``` + +```sh +$ kubectl get memcached/example-memcached -o yaml +apiVersion: cache.example.com/v1alpha1 +kind: Memcached +metadata: + clusterName: "" + creationTimestamp: 2018-03-31T22:51:08Z + generation: 0 + name: example-memcached + namespace: default + resourceVersion: "245453" + selfLink: /apis/cache.example.com/v1alpha1/namespaces/default/memcacheds/example-memcached + uid: 0026cc97-3536-11e8-bd83-0800274106a1 +spec: + size: 3 +status: + nodes: + - example-memcached-6fd7c98d8-7dqdr + - example-memcached-6fd7c98d8-g5k7v + - example-memcached-6fd7c98d8-m7vn7 +``` + +### Update the size + +Change the `spec.size` field in the memcached CR from 3 to 4 and apply the change: + +```sh +$ cat deploy/cr.yaml +apiVersion: "cache.example.com/v1alpha1" +kind: "Memcached" +metadata: + name: "example-memcached" +spec: + size: 4 + +$ kubectl apply -f deploy/cr.yaml +``` + +Confirm that the operator changes the deployment size: + +```sh +$ kubectl get deployment +NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE +example-memcached 4 4 4 4 5m +``` + +### Cleanup + +Clean up the resources: + +```sh +$ kubectl delete -f deploy/cr.yaml +$ kubectl delete -f deploy/operator.yaml +``` + + +## Advanced Topics +### Adding 3rd Party Resources To Your Operator +To add a resource to an operator, you must add it to a scheme. By creating an `AddToScheme` method or reusing one you can easily add a resource to your scheme. An [example][deployments_register] shows that you define a function and then use the [runtime][runtime_package] package to create a `SchemeBuilder` + +#### Current Operator-SDK +You then need to tell the operators to use these functions to add the resources to its scheme. In operator-sdk you use [AddToSDKScheme][osdk_add_to_scheme] to add this. +Example of you main.go: +```go +import ( + .... + appsv1 "k8s.io/api/apps/v1" +) + +func main() { + k8sutil.AddToSDKScheme(appsv1.AddToScheme)` + sdk.Watch(appsv1.SchemeGroupVersion.String(), "Deployments", , ) +} +``` + +#### Future with Controller Runtime +When using controller runtime, you will also need to tell its scheme about your resourece. In controller runtime to add to the scheme, you can get the managers [scheme][manager_scheme]. If you would like to see what kubebuilder generates to add the resoureces to the [scheme][simple_resource]. +Example: +```go +import ( + .... + appsv1 "k8s.io/api/apps/v1" +) + +func main() { + .... + if err := appsv1.AddToScheme(mgr.GetScheme()); err != nil { + log.Fatal(err) + } + .... +} +``` + +[memcached_handler]: ../example/memcached-operator/handler.go.tmpl +[layout_doc]:./project_layout.md +[dep_tool]:https://golang.github.io/dep/docs/installation.html +[git_tool]:https://git-scm.com/downloads +[go_tool]:https://golang.org/dl/ +[docker_tool]:https://docs.docker.com/install/ +[kubectl_tool]:https://kubernetes.io/docs/tasks/tools/install-kubectl/ +[minikube_tool]:https://github.com/kubernetes/minikube#installation +[manager_scheme]: https://github.com/kubernetes-sigs/controller-runtime/blob/master/pkg/manager/manager.go#L61 +[simple_resource]: https://book.kubebuilder.io/basics/simple_resource.html +[deployments_register]: https://github.com/kubernetes/api/blob/master/apps/v1/register.go#L41 +[runtime_package]: https://godoc.org/k8s.io/apimachinery/pkg/runtime +[osdk_add_to_scheme]: https://github.com/operator-framework/operator-sdk/blob/4179b6ac459b2b0cb04ab3a1b438c280bd28d1a5/pkg/util/k8sutil/k8sutil.go#L67 From 8e0de9441a731d1159ba3b81cf5106a0946b64ba Mon Sep 17 00:00:00 2001 From: Dylan Murray Date: Tue, 2 Oct 2018 12:04:00 -0400 Subject: [PATCH 02/14] Add deployment information --- doc/ansible/user-guide.md | 109 +++++++++++++++----------------------- 1 file changed, 42 insertions(+), 67 deletions(-) diff --git a/doc/ansible/user-guide.md b/doc/ansible/user-guide.md index e89221a00a..08dec8134e 100644 --- a/doc/ansible/user-guide.md +++ b/doc/ansible/user-guide.md @@ -95,6 +95,47 @@ First, set a default in case the user doesn't set the `spec` field by modifying size: 1 ``` +### Defining the Memcached deployment + +Now that we have the spec defined, we can define what Ansible is actually executed on resource changes. Since this is an Ansible Role, the default behavior will be to execute the tasks in `roles/Memcached/tasks/main.yml`. We want Ansible to create a deployment if it does not exist which runs the `memcached:1.4.36-alpine` image. Ansible 2.5+ supports the [k8s Ansible Module](https://docs.ansible.com/ansible/2.6/modules/k8s_module.html) which we will leverage to control the deployment definition. + +Modify `roles/Memcached/tasks/main.yml` to look like the following: +```yaml +--- +- name: start memcached + k8s: + definition: + kind: Deployment + apiVersion: apps/v1 + metadata: + name: '{{ meta.name }}-memcached' + namespace: '{{ meta.namespace }}' + spec: + replicas: {{size}} + selector: + matchLabels: + app: memcached + template: + metadata: + labels: + app: memcached + spec: + containers: + - name: memcached + command: + - memcached + - -m=64 + - -o + - modern + - -v + image: "memcached:1.4.36-alpine" + ports: + - containerPort: 11211 + +``` + +It is important to note that we used the `size` variable to control how many replicas of the Memcached deployment we want. We set the default to `1` but any user can create a Custom Resource which overwrites the default. + ### Build and run the operator Before running the operator, Kubernetes needs to know about the new custom resource definition the operator will be watching. @@ -185,7 +226,7 @@ memcached-operator 1 1 1 1 2m example-memcached 3 3 3 3 1m ``` -Check the pods and CR status to confirm the status is updated with the memcached pod names: +Check the pods to confirm 3 replicas were created: ```sh $ kubectl get pods @@ -196,28 +237,6 @@ example-memcached-6fd7c98d8-m7vn7 1/1 Running 0 1m memcached-operator-7cc7cfdf86-vvjqk 1/1 Running 0 2m ``` -```sh -$ kubectl get memcached/example-memcached -o yaml -apiVersion: cache.example.com/v1alpha1 -kind: Memcached -metadata: - clusterName: "" - creationTimestamp: 2018-03-31T22:51:08Z - generation: 0 - name: example-memcached - namespace: default - resourceVersion: "245453" - selfLink: /apis/cache.example.com/v1alpha1/namespaces/default/memcacheds/example-memcached - uid: 0026cc97-3536-11e8-bd83-0800274106a1 -spec: - size: 3 -status: - nodes: - - example-memcached-6fd7c98d8-7dqdr - - example-memcached-6fd7c98d8-g5k7v - - example-memcached-6fd7c98d8-m7vn7 -``` - ### Update the size Change the `spec.size` field in the memcached CR from 3 to 4 and apply the change: @@ -251,45 +270,6 @@ $ kubectl delete -f deploy/cr.yaml $ kubectl delete -f deploy/operator.yaml ``` - -## Advanced Topics -### Adding 3rd Party Resources To Your Operator -To add a resource to an operator, you must add it to a scheme. By creating an `AddToScheme` method or reusing one you can easily add a resource to your scheme. An [example][deployments_register] shows that you define a function and then use the [runtime][runtime_package] package to create a `SchemeBuilder` - -#### Current Operator-SDK -You then need to tell the operators to use these functions to add the resources to its scheme. In operator-sdk you use [AddToSDKScheme][osdk_add_to_scheme] to add this. -Example of you main.go: -```go -import ( - .... - appsv1 "k8s.io/api/apps/v1" -) - -func main() { - k8sutil.AddToSDKScheme(appsv1.AddToScheme)` - sdk.Watch(appsv1.SchemeGroupVersion.String(), "Deployments", , ) -} -``` - -#### Future with Controller Runtime -When using controller runtime, you will also need to tell its scheme about your resourece. In controller runtime to add to the scheme, you can get the managers [scheme][manager_scheme]. If you would like to see what kubebuilder generates to add the resoureces to the [scheme][simple_resource]. -Example: -```go -import ( - .... - appsv1 "k8s.io/api/apps/v1" -) - -func main() { - .... - if err := appsv1.AddToScheme(mgr.GetScheme()); err != nil { - log.Fatal(err) - } - .... -} -``` - -[memcached_handler]: ../example/memcached-operator/handler.go.tmpl [layout_doc]:./project_layout.md [dep_tool]:https://golang.github.io/dep/docs/installation.html [git_tool]:https://git-scm.com/downloads @@ -297,8 +277,3 @@ func main() { [docker_tool]:https://docs.docker.com/install/ [kubectl_tool]:https://kubernetes.io/docs/tasks/tools/install-kubectl/ [minikube_tool]:https://github.com/kubernetes/minikube#installation -[manager_scheme]: https://github.com/kubernetes-sigs/controller-runtime/blob/master/pkg/manager/manager.go#L61 -[simple_resource]: https://book.kubebuilder.io/basics/simple_resource.html -[deployments_register]: https://github.com/kubernetes/api/blob/master/apps/v1/register.go#L41 -[runtime_package]: https://godoc.org/k8s.io/apimachinery/pkg/runtime -[osdk_add_to_scheme]: https://github.com/operator-framework/operator-sdk/blob/4179b6ac459b2b0cb04ab3a1b438c280bd28d1a5/pkg/util/k8sutil/k8sutil.go#L67 From 236ca56a038c63ccae672eeac92b154fd974db47 Mon Sep 17 00:00:00 2001 From: Dylan Murray Date: Tue, 2 Oct 2018 12:41:02 -0400 Subject: [PATCH 03/14] Project layout page --- doc/ansible/project_layout.md | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/doc/ansible/project_layout.md b/doc/ansible/project_layout.md index 031f1c8274..7d18db43e0 100644 --- a/doc/ansible/project_layout.md +++ b/doc/ansible/project_layout.md @@ -1,19 +1,15 @@ # Project Scaffolding Layout After creating a new operator project using -`operator-sdk new`, the project directory has numerous generated folders and files. The following table describes a basic rundown of each generated file/directory. +`operator-sdk new --type ansible`, the project directory has numerous generated folders and files. The following table describes a basic rundown of each generated file/directory. | File/Folders | Purpose | | :--- | :--- | -| Gopkg.toml Gopkg.lock | The [Go Dep][dep] manifests that describe the external dependencies of this operator. | -| cmd | Contains `main.go` which is the entry point to initialize and start this operator using the operator-sdk APIs. | -| config | Contains metadata about state of this project such as project name, kind, api-version, and so forth. The operator-sdk commands use this metadata to perform actions that require knowing the state. | | deploy | Contains a generic set of kubernetes manifests for deploying this operator on a kubernetes cluster. | -| pkg/apis | Contains the directory tree that defines the APIs and types of Custom Resource Definitions(CRD). These files allow the sdk to do code generation for CRD types and register the schemes for all types in order to correctly decode Custom Resource objects. | -| pkg/stub | Contains `handler.go` which is the place for a user to write all the operating business logic. | -| tmp | Contains scripts that the operator-sdk uses for build and code generation. | -| vendor | The golang [vendor][Vendor] folder that contains the local copies of the external dependencies that satisfy the imports of this project. [Go Dep][dep] manages the vendor directly. | +| roles/ | Contains an Ansible Role initialized using [Ansible Galaxy](https://docs.ansible.com/ansible/latest/reference_appendices/galaxy.html) | +| tmp | Contains scripts that the operator-sdk uses for build and initialization. | +| watches.yaml | [Vendor]: https://golang.org/cmd/go/#hdr-Vendor_Directories [dep]: https://github.com/golang/dep From 1fa2ec5745dff68f026ced5a3037a444ccff9754 Mon Sep 17 00:00:00 2001 From: Dylan Murray Date: Tue, 2 Oct 2018 12:56:49 -0400 Subject: [PATCH 04/14] Update project layout --- doc/ansible/project_layout.md | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/doc/ansible/project_layout.md b/doc/ansible/project_layout.md index 7d18db43e0..b16eddcabf 100644 --- a/doc/ansible/project_layout.md +++ b/doc/ansible/project_layout.md @@ -9,7 +9,4 @@ After creating a new operator project using | deploy | Contains a generic set of kubernetes manifests for deploying this operator on a kubernetes cluster. | | roles/ | Contains an Ansible Role initialized using [Ansible Galaxy](https://docs.ansible.com/ansible/latest/reference_appendices/galaxy.html) | | tmp | Contains scripts that the operator-sdk uses for build and initialization. | -| watches.yaml | - -[Vendor]: https://golang.org/cmd/go/#hdr-Vendor_Directories -[dep]: https://github.com/golang/dep +| watches.yaml | Contains Group, Version, Kind, and Ansible invocation method. | From 9ee9cf4103ace36df87ff218ca4e2068b94794ae Mon Sep 17 00:00:00 2001 From: Dylan Murray Date: Tue, 2 Oct 2018 17:14:29 -0400 Subject: [PATCH 05/14] Some small updates --- doc/ansible/user-guide.md | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/doc/ansible/user-guide.md b/doc/ansible/user-guide.md index 08dec8134e..3f4d497b16 100644 --- a/doc/ansible/user-guide.md +++ b/doc/ansible/user-guide.md @@ -4,12 +4,12 @@ This guide walks through an example of building a simple memcached-operator powe ## Prerequisites -- [dep][dep_tool] version v0.5.0+. - [git][git_tool] -- [go][go_tool] version v1.10+. - [docker][docker_tool] version 17.03+. - [kubectl][kubectl_tool] version v1.9.0+. - [ansible][ansible_tool] version v2.6.0+ +- [dep][dep_tool] version v0.5.0+. (Optional if you aren't installing from source) +- [go][go_tool] version v1.10+. (Optional if you aren't installing from source) - Access to a kubernetes v.1.9.0+ cluster. **Note**: This guide uses [minikube][minikube_tool] version v0.25.0+ as the local kubernetes cluster and quay.io for the public registry. @@ -49,9 +49,9 @@ To learn more about the project directory structure, see [project layout][layout ## Customize the operator logic -For this example the memcached-operator will execute the following reconciliation logic for each `Memcached` CR: +For this example the memcached-operator will execute the following reconciliation logic for each `Memcached` Custom Resource (CR): - Create a memcached Deployment if it doesn't exist -- Ensure that the Deployment size is the same as specified by the `Memcached` CR spec +- Ensure that the Deployment size is the same as specified by the `Memcached` CR ### Watch the Memcached CR @@ -86,6 +86,9 @@ Specifying a `playbook` option in `watches.yaml` will configure the operator to ``` ## Building the Memcached Ansible Role + +The first thing to do is to modify the generated Ansible role under `roles/Memcached`. This Ansible Role controls the logic that is executed when a resource is modified. + ### Define the Memcached spec Defining the spec for an Ansible Operator can be done entirely in Ansible. The Ansible Operator will simply pass all key value pairs listed in the Custom Resource spec field along to Ansible as [variables](https://docs.ansible.com/ansible/2.5/user_guide/playbooks_variables.html#passing-variables-on-the-command-line). It is recommended that you perform some type validation in Ansible on the variables to ensure that your application is receiving expected input. @@ -148,12 +151,12 @@ $ kubectl create -f deploy/crd.yaml Once this is done, there are two ways to run the operator: -- As pod inside Kubernetes cluster -- As go program outside cluster using `operator-sdk` +- As a pod inside a Kubernetes cluster +- As a go program outside cluster using `operator-sdk` -#### 1. Run as pod inside a Kubernetes cluster +#### 1. Run as a pod inside a Kubernetes cluster -Run as pod inside a Kubernetes cluster is preferred for production use. +Running as a pod inside a Kubernetes cluster is preferred for production use. Build the memcached-operator image and push it to a registry: ``` @@ -181,7 +184,7 @@ memcached-operator 1 1 1 1 1m #### 2. Run outside the cluster -This method is preferred during development cycle to deploy and test faster. +This method is preferred during the development cycle to speed up deployment and testing. Run the operator locally with the default kubernetes config file present at `$HOME/.kube/config`: @@ -277,3 +280,4 @@ $ kubectl delete -f deploy/operator.yaml [docker_tool]:https://docs.docker.com/install/ [kubectl_tool]:https://kubernetes.io/docs/tasks/tools/install-kubectl/ [minikube_tool]:https://github.com/kubernetes/minikube#installation +[ansible_tool]:https://docs.ansible.com/ansible/latest/index.html From f1e33d239d1f13f40fc2abc3f8bb9bf2690acb4d Mon Sep 17 00:00:00 2001 From: Dylan Murray Date: Tue, 2 Oct 2018 17:52:08 -0400 Subject: [PATCH 06/14] Add link to Ansible guide from main user guide --- doc/user-guide.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/doc/user-guide.md b/doc/user-guide.md index 4030df84f5..094f762038 100644 --- a/doc/user-guide.md +++ b/doc/user-guide.md @@ -2,6 +2,8 @@ This guide walks through an example of building a simple memcached-operator using tools and libraries provided by the Operator SDK. +This guide covers the workflow of creating and deploying an operator written in Go. See the [Ansible User Guide](ansible_user_guide). + ## Prerequisites - [dep][dep_tool] version v0.5.0+. @@ -306,6 +308,7 @@ func main() { [memcached_handler]: ../example/memcached-operator/handler.go.tmpl [layout_doc]:./project_layout.md +[ansible_user_guide]:./ansible/user-guide.md [dep_tool]:https://golang.github.io/dep/docs/installation.html [git_tool]:https://git-scm.com/downloads [go_tool]:https://golang.org/dl/ From c92390e1e57f2a265c27b0a28d79ad8a3e6c55be Mon Sep 17 00:00:00 2001 From: Dylan Murray Date: Tue, 2 Oct 2018 17:53:50 -0400 Subject: [PATCH 07/14] Update link --- doc/user-guide.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/user-guide.md b/doc/user-guide.md index 094f762038..7680bbb794 100644 --- a/doc/user-guide.md +++ b/doc/user-guide.md @@ -2,7 +2,7 @@ This guide walks through an example of building a simple memcached-operator using tools and libraries provided by the Operator SDK. -This guide covers the workflow of creating and deploying an operator written in Go. See the [Ansible User Guide](ansible_user_guide). +This guide covers the workflow of creating and deploying an operator written in Go. To read about the Ansible Operator, see the [Ansible Operator User Guide][ansible_user_guide]. ## Prerequisites From 6ce99ebb36d96a0623cdd7ca042c17c7d81a6402 Mon Sep 17 00:00:00 2001 From: Dylan Murray Date: Tue, 2 Oct 2018 18:08:00 -0400 Subject: [PATCH 08/14] Add CLI reference commands --- doc/sdk-cli-reference.md | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/doc/sdk-cli-reference.md b/doc/sdk-cli-reference.md index 6ef5eac68d..c46e4c10db 100644 --- a/doc/sdk-cli-reference.md +++ b/doc/sdk-cli-reference.md @@ -113,6 +113,29 @@ Run code-generation for custom resources Generating deepcopy funcs ``` +#### crd - Generates a custom resource definition (CRD) and the custom resource (CR) files + +##### Use + +crd generator generates custom resource definition and custom resource +files for the specified api-version and kind. + +##### Flags + +* `--api-version` **(required)** string - Kubernetes apiVersion and has a format of $GROUP_NAME/$VERSION (e.g app.example.com/v1alpha1) +* `-h, --help` - help for k8s +* `--kind` **(required)** string - Kubernetes CustomResourceDefinition kind. (e.g AppService) + +##### Example + +```bash +operator-sdk generate crd --api-version app.example.com/v1alpha1 --kind AppService + +# Output: +Generating custom resource definition (CRD) file +Create /deploy/appservice_cr.yaml +Create /deploy/appservice_crd.yaml +``` #### olm-catalog - Generates OLM Catalog manifests ##### Flags @@ -147,6 +170,8 @@ generates a default directory layout based on the input `project-name`. * `--api-version` **(required)** string - Kubernetes apiVersion and has a format of `$GROUP_NAME/$VERSION` (e.g app.example.com/v1alpha1) * `--kind` **(required)** string - Kubernetes CustomResourceDefintion kind. (e.g AppService) +* `--skip-git-init` Do not init the directory as a git repository +* `--type` Type of operator to initialize (e.g "ansible") (default "go") * `-h, --help` - help for new ### Example From b7d900fdf9449c884026b0e4f23eec9d2576b48d Mon Sep 17 00:00:00 2001 From: Dylan Murray Date: Wed, 3 Oct 2018 10:54:58 -0400 Subject: [PATCH 09/14] Fix ansible typo --- doc/ansible/user-guide.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/ansible/user-guide.md b/doc/ansible/user-guide.md index 3f4d497b16..da19798021 100644 --- a/doc/ansible/user-guide.md +++ b/doc/ansible/user-guide.md @@ -114,7 +114,7 @@ Modify `roles/Memcached/tasks/main.yml` to look like the following: name: '{{ meta.name }}-memcached' namespace: '{{ meta.namespace }}' spec: - replicas: {{size}} + replicas: "{{size}}" selector: matchLabels: app: memcached From 419ecde1f08e78e2e7f3204d71851ba1ff94e1fb Mon Sep 17 00:00:00 2001 From: Dylan Murray Date: Wed, 3 Oct 2018 11:53:49 -0400 Subject: [PATCH 10/14] Mention modifying rbac.yaml --- doc/ansible/user-guide.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/ansible/user-guide.md b/doc/ansible/user-guide.md index da19798021..e85291a979 100644 --- a/doc/ansible/user-guide.md +++ b/doc/ansible/user-guide.md @@ -174,6 +174,8 @@ $ kubectl create -f deploy/rbac.yaml $ kubectl create -f deploy/operator.yaml ``` +**NOTE**: `deploy/rbac.yaml` creates a `ClusterRoleBinding` and assumes we are working in namespace `default`. If you are working in a different namespace you must modify this file before creating it. + Verify that the memcached-operator is up and running: ```sh From a1c1e37f33e1a6593516ea2641554d15252a8754 Mon Sep 17 00:00:00 2001 From: Dylan Murray Date: Thu, 4 Oct 2018 12:48:22 -0400 Subject: [PATCH 11/14] Include ansible-runner information --- doc/ansible/user-guide.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/doc/ansible/user-guide.md b/doc/ansible/user-guide.md index e85291a979..70598de8cb 100644 --- a/doc/ansible/user-guide.md +++ b/doc/ansible/user-guide.md @@ -8,6 +8,8 @@ This guide walks through an example of building a simple memcached-operator powe - [docker][docker_tool] version 17.03+. - [kubectl][kubectl_tool] version v1.9.0+. - [ansible][ansible_tool] version v2.6.0+ +- [ansible-runner][ansible_runner_tool] version v1.1.0+ +- [ansible-runner-http][ansible_runner_http_plugin] version v1.0.0+ - [dep][dep_tool] version v0.5.0+. (Optional if you aren't installing from source) - [go][go_tool] version v1.10+. (Optional if you aren't installing from source) - Access to a kubernetes v.1.9.0+ cluster. @@ -188,6 +190,9 @@ memcached-operator 1 1 1 1 1m This method is preferred during the development cycle to speed up deployment and testing. +**Note**: Ensure that [Ansible Runner][ansible_runner_tool] and [Ansible Runner HTTP Plugin][ansible_runner_http_plugin] is installed or else you will see unexpected errors from Ansible Runner when a Custom Resource is created. + +It is also important that the `role` path referenced in `watches.yaml` exists on your machine. Since we are normally used to using a container where the Role is put on disk for us, we need to manually copy our role to a path expected by Ansible. Run the operator locally with the default kubernetes config file present at `$HOME/.kube/config`: ```sh @@ -283,3 +288,5 @@ $ kubectl delete -f deploy/operator.yaml [kubectl_tool]:https://kubernetes.io/docs/tasks/tools/install-kubectl/ [minikube_tool]:https://github.com/kubernetes/minikube#installation [ansible_tool]:https://docs.ansible.com/ansible/latest/index.html +[ansible_runner_tool]:https://ansible-runner.readthedocs.io/en/latest/install.html +[ansible_runner_http_plugin]:https://github.com/ansible/ansible-runner-http From 38930019260a6ca0ec9ef58cdc8b225f384f03db Mon Sep 17 00:00:00 2001 From: Dylan Murray Date: Thu, 4 Oct 2018 15:32:45 -0400 Subject: [PATCH 12/14] Add some changes for up local --- doc/ansible/user-guide.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/doc/ansible/user-guide.md b/doc/ansible/user-guide.md index 70598de8cb..78dcc119de 100644 --- a/doc/ansible/user-guide.md +++ b/doc/ansible/user-guide.md @@ -163,11 +163,13 @@ Running as a pod inside a Kubernetes cluster is preferred for production use. Build the memcached-operator image and push it to a registry: ``` $ operator-sdk build quay.io/example/memcached-operator:v0.0.1 -$ sed -i 's|REPLACE_IMAGE|quay.io/example/memcached-operator:v0.0.1|g' deploy/operator.yaml $ docker push quay.io/example/memcached-operator:v0.0.1 ``` -Kubernetes deployment manifests are generated in `deploy/operator.yaml`. The deployment image is set to the container image specified above. +Kubernetes deployment manifests are generated in `deploy/operator.yaml`. The deployment image in this file needs to be modified from the placeholder `REPLACE_IMAGE` to the previous built image. To do this run: +``` +$ sed -i 's|REPLACE_IMAGE|quay.io/example/memcached-operator:v0.0.1|g' deploy/operator.yaml +``` Deploy the memcached-operator: @@ -192,7 +194,8 @@ This method is preferred during the development cycle to speed up deployment and **Note**: Ensure that [Ansible Runner][ansible_runner_tool] and [Ansible Runner HTTP Plugin][ansible_runner_http_plugin] is installed or else you will see unexpected errors from Ansible Runner when a Custom Resource is created. -It is also important that the `role` path referenced in `watches.yaml` exists on your machine. Since we are normally used to using a container where the Role is put on disk for us, we need to manually copy our role to a path expected by Ansible. +It is also important that the `role` path referenced in `watches.yaml` exists on your machine. Since we are normally used to using a container where the Role is put on disk for us, we need to manually copy our role to the configured Ansible Roles path (e.g `/etc/ansible/roles`. + Run the operator locally with the default kubernetes config file present at `$HOME/.kube/config`: ```sh From 04c64109096a5c2078c6cc2b99bed99f0a0ea28b Mon Sep 17 00:00:00 2001 From: Dylan Murray Date: Fri, 5 Oct 2018 09:03:58 -0400 Subject: [PATCH 13/14] Enforce 80 char limit and grammar updates --- doc/ansible/user-guide.md | 88 +++++++++++++++++++++++++++++---------- doc/user-guide.md | 4 +- 2 files changed, 68 insertions(+), 24 deletions(-) diff --git a/doc/ansible/user-guide.md b/doc/ansible/user-guide.md index 78dcc119de..57c089554f 100644 --- a/doc/ansible/user-guide.md +++ b/doc/ansible/user-guide.md @@ -1,6 +1,7 @@ # User Guide -This guide walks through an example of building a simple memcached-operator powered by Ansible using tools and libraries provided by the Operator SDK. +This guide walks through an example of building a simple memcached-operator +powered by Ansible using tools and libraries provided by the Operator SDK. ## Prerequisites @@ -14,11 +15,13 @@ This guide walks through an example of building a simple memcached-operator powe - [go][go_tool] version v1.10+. (Optional if you aren't installing from source) - Access to a kubernetes v.1.9.0+ cluster. -**Note**: This guide uses [minikube][minikube_tool] version v0.25.0+ as the local kubernetes cluster and quay.io for the public registry. +**Note**: This guide uses [minikube][minikube_tool] version v0.25.0+ as the +local kubernetes cluster and quay.io for the public registry. ## Install the Operator SDK CLI -The Operator SDK has a CLI tool that helps the developer to create, build, and deploy a new operator project. +The Operator SDK has a CLI tool that helps the developer to create, build, and +deploy a new operator project. Checkout the desired release tag and install the SDK CLI tool: @@ -45,19 +48,25 @@ $ operator-sdk new memcached-operator --api-version=cache.example.com/v1alpha1 - $ cd memcached-operator ``` -This creates the memcached-operator project specifically for watching the Memcached resource with APIVersion `cache.example.com/v1apha1` and Kind `Memcached`. +This creates the memcached-operator project specifically for watching the +Memcached resource with APIVersion `cache.example.com/v1apha1` and Kind +`Memcached`. -To learn more about the project directory structure, see [project layout][layout_doc] doc. +To learn more about the project directory structure, see [project +layout][layout_doc] doc. ## Customize the operator logic -For this example the memcached-operator will execute the following reconciliation logic for each `Memcached` Custom Resource (CR): +For this example the memcached-operator will execute the following +reconciliation logic for each `Memcached` Custom Resource (CR): - Create a memcached Deployment if it doesn't exist -- Ensure that the Deployment size is the same as specified by the `Memcached` CR +- Ensure that the Deployment size is the same as specified by the `Memcached` +CR ### Watch the Memcached CR -By default, the memcached-operator watches `Memcached` resource events as shown in `watches.yaml` and executes Ansible Role `Memached`: +By default, the memcached-operator watches `Memcached` resource events as shown +in `watches.yaml` and executes Ansible Role `Memached`: ```yaml --- @@ -68,7 +77,10 @@ By default, the memcached-operator watches `Memcached` resource events as shown #### Options **Role** -Specifying a `role` option in `watches.yaml` will configure the operator to use this specified path when launching `ansible-runner` with an Ansible Role. This is the default. +Specifying a `role` option in `watches.yaml` will configure the operator to use +this specified path when launching `ansible-runner` with an Ansible Role. By +default, the `new` command will fill in an absolute path to where your role +should go. ```yaml --- - version: v1alpha1 @@ -78,7 +90,9 @@ Specifying a `role` option in `watches.yaml` will configure the operator to use ``` **Playbook** -Specifying a `playbook` option in `watches.yaml` will configure the operator to use this specified path when launching `ansible-runner` with an Ansible Playbook +Specifying a `playbook` option in `watches.yaml` will configure the operator to +use this specified path when launching `ansible-runner` with an Ansible +Playbook ```yaml --- - version: v1alpha1 @@ -89,20 +103,34 @@ Specifying a `playbook` option in `watches.yaml` will configure the operator to ## Building the Memcached Ansible Role -The first thing to do is to modify the generated Ansible role under `roles/Memcached`. This Ansible Role controls the logic that is executed when a resource is modified. +The first thing to do is to modify the generated Ansible role under +`roles/Memcached`. This Ansible Role controls the logic that is executed when a +resource is modified. ### Define the Memcached spec -Defining the spec for an Ansible Operator can be done entirely in Ansible. The Ansible Operator will simply pass all key value pairs listed in the Custom Resource spec field along to Ansible as [variables](https://docs.ansible.com/ansible/2.5/user_guide/playbooks_variables.html#passing-variables-on-the-command-line). It is recommended that you perform some type validation in Ansible on the variables to ensure that your application is receiving expected input. +Defining the spec for an Ansible Operator can be done entirely in Ansible. The +Ansible Operator will simply pass all key value pairs listed in the Custom +Resource spec field along to Ansible as +[variables](https://docs.ansible.com/ansible/2.5/user_guide/playbooks_variables.html#passing-variables-on-the-command-line). +It is recommended that you perform some type validation in Ansible on the +variables to ensure that your application is receiving expected input. -First, set a default in case the user doesn't set the `spec` field by modifying `roles/Memcached/defaults/main.yml`: +First, set a default in case the user doesn't set the `spec` field by modifying +`roles/Memcached/defaults/main.yml`: ```yaml size: 1 ``` ### Defining the Memcached deployment -Now that we have the spec defined, we can define what Ansible is actually executed on resource changes. Since this is an Ansible Role, the default behavior will be to execute the tasks in `roles/Memcached/tasks/main.yml`. We want Ansible to create a deployment if it does not exist which runs the `memcached:1.4.36-alpine` image. Ansible 2.5+ supports the [k8s Ansible Module](https://docs.ansible.com/ansible/2.6/modules/k8s_module.html) which we will leverage to control the deployment definition. +Now that we have the spec defined, we can define what Ansible is actually +executed on resource changes. Since this is an Ansible Role, the default +behavior will be to execute the tasks in `roles/Memcached/tasks/main.yml`. We +want Ansible to create a deployment if it does not exist which runs the +`memcached:1.4.36-alpine` image. Ansible 2.5+ supports the [k8s Ansible +Module](https://docs.ansible.com/ansible/2.6/modules/k8s_module.html) which we +will leverage to control the deployment definition. Modify `roles/Memcached/tasks/main.yml` to look like the following: ```yaml @@ -139,11 +167,14 @@ Modify `roles/Memcached/tasks/main.yml` to look like the following: ``` -It is important to note that we used the `size` variable to control how many replicas of the Memcached deployment we want. We set the default to `1` but any user can create a Custom Resource which overwrites the default. +It is important to note that we used the `size` variable to control how many +replicas of the Memcached deployment we want. We set the default to `1`, but +any user can create a Custom Resource that overwrites the default. ### Build and run the operator -Before running the operator, Kubernetes needs to know about the new custom resource definition the operator will be watching. +Before running the operator, Kubernetes needs to know about the new custom +resource definition the operator will be watching. Deploy the CRD: @@ -154,7 +185,7 @@ $ kubectl create -f deploy/crd.yaml Once this is done, there are two ways to run the operator: - As a pod inside a Kubernetes cluster -- As a go program outside cluster using `operator-sdk` +- As a go program outside the cluster using `operator-sdk` #### 1. Run as a pod inside a Kubernetes cluster @@ -166,7 +197,9 @@ $ operator-sdk build quay.io/example/memcached-operator:v0.0.1 $ docker push quay.io/example/memcached-operator:v0.0.1 ``` -Kubernetes deployment manifests are generated in `deploy/operator.yaml`. The deployment image in this file needs to be modified from the placeholder `REPLACE_IMAGE` to the previous built image. To do this run: +Kubernetes deployment manifests are generated in `deploy/operator.yaml`. The +deployment image in this file needs to be modified from the placeholder +`REPLACE_IMAGE` to the previous built image. To do this run: ``` $ sed -i 's|REPLACE_IMAGE|quay.io/example/memcached-operator:v0.0.1|g' deploy/operator.yaml ``` @@ -178,7 +211,9 @@ $ kubectl create -f deploy/rbac.yaml $ kubectl create -f deploy/operator.yaml ``` -**NOTE**: `deploy/rbac.yaml` creates a `ClusterRoleBinding` and assumes we are working in namespace `default`. If you are working in a different namespace you must modify this file before creating it. +**NOTE**: `deploy/rbac.yaml` creates a `ClusterRoleBinding` and assumes we are +working in namespace `default`. If you are working in a different namespace you +must modify this file before creating it. Verify that the memcached-operator is up and running: @@ -192,11 +227,17 @@ memcached-operator 1 1 1 1 1m This method is preferred during the development cycle to speed up deployment and testing. -**Note**: Ensure that [Ansible Runner][ansible_runner_tool] and [Ansible Runner HTTP Plugin][ansible_runner_http_plugin] is installed or else you will see unexpected errors from Ansible Runner when a Custom Resource is created. +**Note**: Ensure that [Ansible Runner][ansible_runner_tool] and [Ansible Runner +HTTP Plugin][ansible_runner_http_plugin] is installed or else you will see +unexpected errors from Ansible Runner when a Custom Resource is created. -It is also important that the `role` path referenced in `watches.yaml` exists on your machine. Since we are normally used to using a container where the Role is put on disk for us, we need to manually copy our role to the configured Ansible Roles path (e.g `/etc/ansible/roles`. +It is also important that the `role` path referenced in `watches.yaml` exists +on your machine. Since we are normally used to using a container where the Role +is put on disk for us, we need to manually copy our role to the configured +Ansible Roles path (e.g `/etc/ansible/roles`. -Run the operator locally with the default kubernetes config file present at `$HOME/.kube/config`: +Run the operator locally with the default kubernetes config file present at +`$HOME/.kube/config`: ```sh $ operator-sdk up local @@ -252,7 +293,8 @@ memcached-operator-7cc7cfdf86-vvjqk 1/1 Running 0 2m ### Update the size -Change the `spec.size` field in the memcached CR from 3 to 4 and apply the change: +Change the `spec.size` field in the memcached CR from 3 to 4 and apply the +change: ```sh $ cat deploy/cr.yaml diff --git a/doc/user-guide.md b/doc/user-guide.md index 7680bbb794..3475065b74 100644 --- a/doc/user-guide.md +++ b/doc/user-guide.md @@ -2,7 +2,9 @@ This guide walks through an example of building a simple memcached-operator using tools and libraries provided by the Operator SDK. -This guide covers the workflow of creating and deploying an operator written in Go. To read about the Ansible Operator, see the [Ansible Operator User Guide][ansible_user_guide]. +To learn how to use Ansible to create a Memcached operator, see [Ansible +Operator User Guide][ansible_user_guide]. The rest of this document will show +how to program an operator in Go. ## Prerequisites From 9dc72065459cd1bc66b6b84a732ef32b7610c6b7 Mon Sep 17 00:00:00 2001 From: Dylan Murray Date: Fri, 5 Oct 2018 10:57:13 -0400 Subject: [PATCH 14/14] Fully qualify image name --- doc/ansible/user-guide.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/ansible/user-guide.md b/doc/ansible/user-guide.md index 57c089554f..112098c420 100644 --- a/doc/ansible/user-guide.md +++ b/doc/ansible/user-guide.md @@ -161,7 +161,7 @@ Modify `roles/Memcached/tasks/main.yml` to look like the following: - -o - modern - -v - image: "memcached:1.4.36-alpine" + image: "docker.io/memcached:1.4.36-alpine" ports: - containerPort: 11211