From 1f8510d83be07aad87f9fe738647b7ebc1abc282 Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Fri, 9 Jul 2021 13:31:07 +0200 Subject: [PATCH 01/20] runtime: outline package and goals in README Signed-off-by: Hidde Beydals --- runtime/README.md | 25 +++++++++++++++++++++++++ runtime/doc.go | 3 +-- 2 files changed, 26 insertions(+), 2 deletions(-) create mode 100644 runtime/README.md diff --git a/runtime/README.md b/runtime/README.md new file mode 100644 index 000000000..06dd500b3 --- /dev/null +++ b/runtime/README.md @@ -0,0 +1,25 @@ +# runtime + +runtime offers a set of standard controller runtime packages that can be used on their own, but are best (and at times, +must be) used together to help with common operations. + +### Goals + +- Provide a better development and review experience while working with a set of controllers by creating simple + APIs for common controller and reconciliation operations, like working with Conditions and Events, and debugging. +- Provide utilities to make it easier to adhere to the + [Kubernetes API conventions](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md) + and a selective set of other Kubernetes (SIG) standards like + [kstatus](https://github.com/kubernetes-sigs/cli-utils/tree/master/pkg/kstatus). +- Prefer adoption of existing standards and types (like + [`metav1.Condition`](https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#Condition)) over creating new ones. +- Provide solutions to common difficulties while performing certain Kubernetes operations as a controller, like + patching. +- Standardise how a controller communicates with the outside world, to improve observation and operation experience. +- Standardise the way controller runtime settings are configured, to improve end-user experience. + +### Non-goals + +- Become a toolbox for all problems, packages must be of interest to a wide range of controllers (and specifically, + their runtime operations) before introduction should be considered. +- Adopt conflicting standards without breaking MAJOR version; opinions with versioning. diff --git a/runtime/doc.go b/runtime/doc.go index 929256810..a2c9c3ddd 100644 --- a/runtime/doc.go +++ b/runtime/doc.go @@ -14,6 +14,5 @@ See the License for the specific language governing permissions and limitations under the License. */ -// Package runtime and subpackages contain generic helpers for use -// with controller-runtime. +// Package runtime and subpackages contain generic helpers for use with controller-runtime. package runtime From f6e27a9bda86bfb570aa99974fc77d1b0303dded Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Fri, 9 Jul 2021 18:48:28 +0200 Subject: [PATCH 02/20] runtime: list supported standards in README Signed-off-by: Hidde Beydals --- runtime/README.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/runtime/README.md b/runtime/README.md index 06dd500b3..12e987078 100644 --- a/runtime/README.md +++ b/runtime/README.md @@ -23,3 +23,11 @@ must be) used together to help with common operations. - Become a toolbox for all problems, packages must be of interest to a wide range of controllers (and specifically, their runtime operations) before introduction should be considered. - Adopt conflicting standards without breaking MAJOR version; opinions with versioning. + +## Supported standards + +The packages build upon the following standards: + +- [Kubernetes API conventions](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md) +- [Kubernetes meta API conditions](https://github.com/kubernetes/enhancements/blob/master/keps/sig-api-machinery/1623-standardize-conditions/README.md) +- [kstatus](https://github.com/kubernetes-sigs/cli-utils/tree/master/pkg/kstatus) From bf3b7618c4b30353410907f75392af132b4ce9b0 Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Fri, 9 Jul 2021 20:00:37 +0200 Subject: [PATCH 03/20] runtime: document usage in README Signed-off-by: Hidde Beydals --- runtime/README.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/runtime/README.md b/runtime/README.md index 12e987078..076b600c5 100644 --- a/runtime/README.md +++ b/runtime/README.md @@ -31,3 +31,11 @@ The packages build upon the following standards: - [Kubernetes API conventions](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md) - [Kubernetes meta API conditions](https://github.com/kubernetes/enhancements/blob/master/keps/sig-api-machinery/1623-standardize-conditions/README.md) - [kstatus](https://github.com/kubernetes-sigs/cli-utils/tree/master/pkg/kstatus) + +## Usage + +To use the packages in your project, import `runtime` using `go get` or your dependency manager of choice: + +```shell +go get github.com/fluxcd/pkg/runtime +``` From 4aa5821022142e913751ccbc36be571d15213111 Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Mon, 12 Jul 2021 09:43:31 +0200 Subject: [PATCH 04/20] runtime: document conditions pacakge in README Signed-off-by: Hidde Beydals --- runtime/README.md | 35 ++++++++++++++++++++++++++++++++++- 1 file changed, 34 insertions(+), 1 deletion(-) diff --git a/runtime/README.md b/runtime/README.md index 076b600c5..4485c3edf 100644 --- a/runtime/README.md +++ b/runtime/README.md @@ -1,5 +1,7 @@ # runtime +[![GoDoc](https://pkg.go.dev/badge/github.com/fluxcd/pkg/runtime?utm_source=godoc)](https://pkg.go.dev/github.com/fluxcd/pkg/runtime) + runtime offers a set of standard controller runtime packages that can be used on their own, but are best (and at times, must be) used together to help with common operations. @@ -34,8 +36,39 @@ The packages build upon the following standards: ## Usage -To use the packages in your project, import `runtime` using `go get` or your dependency manager of choice: +To use the packages in your project, import `github.com/fluxcd/pkg/runtime` using `go get` or your dependency manager +of choice: ```shell go get github.com/fluxcd/pkg/runtime ``` + +### Working with Conditions + +The [`conditions`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/conditions) +package can be used on resources that implement the [`conditions.Getter`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/conditions#Getter) +and/or [`conditions.Setter`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/conditions#Setter) +interface, to enhance the experience of working with Conditions on a Kubernetes resource object during reconcile +operations. + +More specifically, it allows you to: + +- Get a Condition from a Kubernetes resource, or a specific value from a Condition, using + [`conditions.Get`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/conditions#Get) + or one of the other available getter functions like + [`conditions.GetMessage`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/conditions#GetMessage). +- Check if a Kubernetes resource has a Condition of a given type using + [`conditions.Has`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/conditions#Has), + or if it bears a Condition in a certain state, for example with + [`conditions.IsFalse`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/conditions#IsFalse). +- Compose [`metav1.Condition`](https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#Condition) structs in a certain + state using e.g. [`conditions.TrueCondition`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/conditions#TrueCondition) + or [`conditions.FalseCondition`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/conditions#FalseCondition). +- Modify the conditions on a Kubernetes resource object using [`conditions.Set`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/conditions#Set) + or one of the available scoped functions like [`conditions.MarkTrue`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/conditions#MarkTrue). +- Compose conditions based on other state and/or configurations using + [`conditions.SetAggregate`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/conditions#SetAggregate), + [`conditions.SetMirror`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/conditions#SetMirror) + and [`conditions.SetSummary`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/conditions#SetSummary). + +For all available functions, see the [package documentation](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/conditions). From 6cd2168597e57224b1fe1e787c3d85c87324b0b8 Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Mon, 12 Jul 2021 10:30:03 +0200 Subject: [PATCH 05/20] runtime: document runtime config opts in README Signed-off-by: Hidde Beydals --- runtime/README.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/runtime/README.md b/runtime/README.md index 4485c3edf..36528799c 100644 --- a/runtime/README.md +++ b/runtime/README.md @@ -72,3 +72,14 @@ More specifically, it allows you to: and [`conditions.SetSummary`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/conditions#SetSummary). For all available functions, see the [package documentation](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/conditions). + +### Runtime configuration options + +Several packages are available to align common runtime configuration flags across a set of controllers, easing the +end-user operator experience. + +| Package | Description | Package documentation | +|---|---|---| +| `client` | Kubernetes runtime client configurations like QPS and burst | [![GoDoc](https://pkg.go.dev/badge/github.com/fluxcd/pkg/runtime/client?utm_source=godoc)](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/client) | +| `leaderelection` | Kubernetes leader election configurations like the lease duration | [![GoDoc](https://pkg.go.dev/badge/github.com/fluxcd/pkg/runtime/leaderelection?utm_source=godoc)](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/leaderelection) | +| `logger` | Runtime logger configurations like the encoding and log level | [![GoDoc](https://pkg.go.dev/badge/github.com/fluxcd/pkg/runtime/logger?utm_source=godoc)](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/logger) | From 00396211bf4cd74fe49baafed588ed2ecabe7a69 Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Mon, 12 Jul 2021 11:07:03 +0200 Subject: [PATCH 06/20] runtime/pprof: expose Endpoints This makes them part of the public API (and thereby makes them documented) and allows overwrites in case this is ever required. Signed-off-by: Hidde Beydals --- runtime/pprof/pprof.go | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/runtime/pprof/pprof.go b/runtime/pprof/pprof.go index bf6ffaf4c..af2ce3724 100644 --- a/runtime/pprof/pprof.go +++ b/runtime/pprof/pprof.go @@ -25,10 +25,11 @@ import ( ctrl "sigs.k8s.io/controller-runtime" ) -// HTTPPrefixPProf is the prefix appended to all endpoints. +// HTTPPrefixPProf is the prefix appended to all Endpoints. const HTTPPrefixPProf = "/debug/pprof" -var endpoints = map[string]http.Handler{ +// Endpoints defines the debugging endpoints that are added by SetupHandlers. +var Endpoints = map[string]http.Handler{ HTTPPrefixPProf + "/": http.HandlerFunc(pprof.Index), HTTPPrefixPProf + "/cmdline": http.HandlerFunc(pprof.Cmdline), HTTPPrefixPProf + "/profile": http.HandlerFunc(pprof.Profile), @@ -60,7 +61,7 @@ func SetupHandlers(mgr ctrl.Manager, log logr.Logger) { runtime.SetMutexProfileFraction(5) } - for p, h := range endpoints { + for p, h := range Endpoints { if err := mgr.AddMetricsExtraHandler(p, h); err != nil { log.Error(err, "unable to add pprof handler") } From 411b581a31b3c235080872c727334387c5d36aaf Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Mon, 12 Jul 2021 11:19:57 +0200 Subject: [PATCH 07/20] runtime: document debugging in README Signed-off-by: Hidde Beydals --- runtime/README.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/runtime/README.md b/runtime/README.md index 36528799c..968677dc3 100644 --- a/runtime/README.md +++ b/runtime/README.md @@ -83,3 +83,12 @@ end-user operator experience. | `client` | Kubernetes runtime client configurations like QPS and burst | [![GoDoc](https://pkg.go.dev/badge/github.com/fluxcd/pkg/runtime/client?utm_source=godoc)](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/client) | | `leaderelection` | Kubernetes leader election configurations like the lease duration | [![GoDoc](https://pkg.go.dev/badge/github.com/fluxcd/pkg/runtime/leaderelection?utm_source=godoc)](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/leaderelection) | | `logger` | Runtime logger configurations like the encoding and log level | [![GoDoc](https://pkg.go.dev/badge/github.com/fluxcd/pkg/runtime/logger?utm_source=godoc)](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/logger) | + +### Debugging + +The [`pprof`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/pprof) package allows setting up additional [Go `pprof`](https://golang.org/pkg/net/http/pprof/) +HTTP handlers on the metrics endpoint of a controller-runtime manager for debugging purposes. A list of exposed +endpoints can be found in [`pprof.Endpoints`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/pprof#Endpoints). + +See the [package documentation](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/pprof#SetupHandlers) for further +instructions on how to use the package. From 8628d6a17a3f4b4a442634d60f5f6c0e9ab995ef Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Mon, 12 Jul 2021 12:00:59 +0200 Subject: [PATCH 08/20] runtime: document testing in README Signed-off-by: Hidde Beydals --- runtime/README.md | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/runtime/README.md b/runtime/README.md index 968677dc3..0a75bfd50 100644 --- a/runtime/README.md +++ b/runtime/README.md @@ -71,14 +71,14 @@ More specifically, it allows you to: [`conditions.SetMirror`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/conditions#SetMirror) and [`conditions.SetSummary`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/conditions#SetSummary). -For all available functions, see the [package documentation](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/conditions). +For all available functions, see the [package reference](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/conditions). ### Runtime configuration options Several packages are available to align common runtime configuration flags across a set of controllers, easing the end-user operator experience. -| Package | Description | Package documentation | +| Package | Description | Reference | |---|---|---| | `client` | Kubernetes runtime client configurations like QPS and burst | [![GoDoc](https://pkg.go.dev/badge/github.com/fluxcd/pkg/runtime/client?utm_source=godoc)](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/client) | | `leaderelection` | Kubernetes leader election configurations like the lease duration | [![GoDoc](https://pkg.go.dev/badge/github.com/fluxcd/pkg/runtime/leaderelection?utm_source=godoc)](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/leaderelection) | @@ -90,5 +90,17 @@ The [`pprof`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/pprof) package al HTTP handlers on the metrics endpoint of a controller-runtime manager for debugging purposes. A list of exposed endpoints can be found in [`pprof.Endpoints`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/pprof#Endpoints). -See the [package documentation](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/pprof#SetupHandlers) for further +See the [package reference](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/pprof#SetupHandlers) for further instructions on how to use the package. + +### Testing + +The [`testenv`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/testenv) package can be utilized to control the +lifecycle of a local Kubernetes api-server used for testing purposes, and offers a set of helper utilities to +work with resources on the cluster. + +It allows control over the runtime scheme and +Custom Resource Defintions using [`testenv.WithScheme`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/testenv#WithCRDPath) +and [`testenv.WithCRDPath`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/testenv#WithCRDPath). + +For all available functions, see the [package reference](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/testenv). From 666b976895541ddfda1ff451d5560e0e515f4d64 Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Mon, 12 Jul 2021 12:58:24 +0200 Subject: [PATCH 09/20] runtime: document safe patching in README Signed-off-by: Hidde Beydals --- runtime/README.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/runtime/README.md b/runtime/README.md index 0a75bfd50..07325be40 100644 --- a/runtime/README.md +++ b/runtime/README.md @@ -73,6 +73,17 @@ More specifically, it allows you to: For all available functions, see the [package reference](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/conditions). +### Safe patching + +The [`patch`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/patch) package offers a helper utility to safely patch +a Kubernetes resource while taking into account a set of configuration options, and attempting to resolve merge +conflicts and retry before bailing. + +It can be configured to understand "owned" Condition types using [`patch.WithOwnedConditions`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/patch#WithOwnedConditions), +and offers other options like [`patch.WithObservedGeneration`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/patch#WithStatusObservedGeneration). + +For all available functions and examples, see the [package reference](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/patch). + ### Runtime configuration options Several packages are available to align common runtime configuration flags across a set of controllers, easing the From e4fb0a9a45c39bc2e1fd2bafc903d6580f746222 Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Mon, 12 Jul 2021 13:15:57 +0200 Subject: [PATCH 10/20] runtime: document controller helpers in README Signed-off-by: Hidde Beydals --- runtime/README.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/runtime/README.md b/runtime/README.md index 07325be40..f12f5dd02 100644 --- a/runtime/README.md +++ b/runtime/README.md @@ -73,6 +73,17 @@ More specifically, it allows you to: For all available functions, see the [package reference](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/conditions). +### Controller helpers + +The [`controller`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/controller) package offers a collection of helpers +that can be embedded into a reconciler struct to provide them the capabilitiy to e.g. send Kubernetes Events and/or +record metrics. + +| Helper | Description | Reference | +|---|---|---| +| `controller.Events` | Provides the capabilities to send Events to the Kubernetes API and an external event recorder | [![GoDoc](https://pkg.go.dev/badge/github.com/fluxcd/pkg/runtime/controller?utm_source=godoc)](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/controller#Events) | +| `controller.Metrics` | Provides the capabilities to record a set of common Prometheus metrics for a Kubernetes resource object | [![GoDoc](https://pkg.go.dev/badge/github.com/fluxcd/pkg/runtime/controller?utm_source=godoc)](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/controller#Metrics) | + ### Safe patching The [`patch`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/patch) package offers a helper utility to safely patch From 46ccb1185e09c7e0492bf42e9cd3b42b627b9a8d Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Mon, 12 Jul 2021 13:58:20 +0200 Subject: [PATCH 11/20] runtime: document forward events in README Signed-off-by: Hidde Beydals --- runtime/README.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/runtime/README.md b/runtime/README.md index f12f5dd02..995a3ad6e 100644 --- a/runtime/README.md +++ b/runtime/README.md @@ -73,6 +73,15 @@ More specifically, it allows you to: For all available functions, see the [package reference](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/conditions). +### Forwarding Events + +The [`events`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/events) package contains an `events.Recorder` which can +be used to forward events to an external endpoint that understands the `events.Event` payload. For [GitOps +Toolkit](https://fluxcd.io/docs/components/) controllers, this is the notification-controller endpoint. + +The package is best used in combination with the [`controller.Metrics` helper](#Controller-helpers), as this allows you +to record _and_ forward Kubernetes Events using the same API. + ### Controller helpers The [`controller`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/controller) package offers a collection of helpers From 063c42dea629566817312c1a367cbf3d30d1a00c Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Mon, 12 Jul 2021 14:56:21 +0200 Subject: [PATCH 12/20] runtime: document recording metrics in README Signed-off-by: Hidde Beydals --- runtime/README.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/runtime/README.md b/runtime/README.md index 995a3ad6e..c5a909419 100644 --- a/runtime/README.md +++ b/runtime/README.md @@ -82,6 +82,14 @@ Toolkit](https://fluxcd.io/docs/components/) controllers, this is the notificati The package is best used in combination with the [`controller.Metrics` helper](#Controller-helpers), as this allows you to record _and_ forward Kubernetes Events using the same API. +### Recording Metrics + +The [`metrics`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/metrics) package offers a `metrics.Recorder` with a +set of Prometheus collectors for common [GitOps Toolkit](https://fluxcd.io/docs/components/) Kubernetes +resource objects metric points (`gotk_*`). + +The package is best used in combination with the [`controller.Events` helper](#Controller-helpers), as this allows you to record metrics using controller-runtime types. + ### Controller helpers The [`controller`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/controller) package offers a collection of helpers From 6ee5af32e016d7b20fbee62ac3d1adfc447db006 Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Mon, 12 Jul 2021 14:58:35 +0200 Subject: [PATCH 13/20] runtime: change "usage" order in README Signed-off-by: Hidde Beydals --- runtime/README.md | 44 ++++++++++++++++++++++---------------------- 1 file changed, 22 insertions(+), 22 deletions(-) diff --git a/runtime/README.md b/runtime/README.md index c5a909419..d0ba51e8a 100644 --- a/runtime/README.md +++ b/runtime/README.md @@ -43,6 +43,17 @@ of choice: go get github.com/fluxcd/pkg/runtime ``` +### Runtime configuration options + +Several packages are available to align common runtime configuration flags across a set of controllers, easing the +end-user operator experience. + +| Package | Description | Reference | +|---|---|---| +| `client` | Kubernetes runtime client configurations like QPS and burst | [![GoDoc](https://pkg.go.dev/badge/github.com/fluxcd/pkg/runtime/client?utm_source=godoc)](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/client) | +| `leaderelection` | Kubernetes leader election configurations like the lease duration | [![GoDoc](https://pkg.go.dev/badge/github.com/fluxcd/pkg/runtime/leaderelection?utm_source=godoc)](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/leaderelection) | +| `logger` | Runtime logger configurations like the encoding and log level | [![GoDoc](https://pkg.go.dev/badge/github.com/fluxcd/pkg/runtime/logger?utm_source=godoc)](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/logger) | + ### Working with Conditions The [`conditions`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/conditions) @@ -73,6 +84,17 @@ More specifically, it allows you to: For all available functions, see the [package reference](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/conditions). +### Safe patching + +The [`patch`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/patch) package offers a helper utility to safely patch +a Kubernetes resource while taking into account a set of configuration options, and attempting to resolve merge +conflicts and retry before bailing. + +It can be configured to understand "owned" Condition types using [`patch.WithOwnedConditions`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/patch#WithOwnedConditions), +and offers other options like [`patch.WithObservedGeneration`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/patch#WithStatusObservedGeneration). + +For all available functions and examples, see the [package reference](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/patch). + ### Forwarding Events The [`events`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/events) package contains an `events.Recorder` which can @@ -101,28 +123,6 @@ record metrics. | `controller.Events` | Provides the capabilities to send Events to the Kubernetes API and an external event recorder | [![GoDoc](https://pkg.go.dev/badge/github.com/fluxcd/pkg/runtime/controller?utm_source=godoc)](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/controller#Events) | | `controller.Metrics` | Provides the capabilities to record a set of common Prometheus metrics for a Kubernetes resource object | [![GoDoc](https://pkg.go.dev/badge/github.com/fluxcd/pkg/runtime/controller?utm_source=godoc)](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/controller#Metrics) | -### Safe patching - -The [`patch`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/patch) package offers a helper utility to safely patch -a Kubernetes resource while taking into account a set of configuration options, and attempting to resolve merge -conflicts and retry before bailing. - -It can be configured to understand "owned" Condition types using [`patch.WithOwnedConditions`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/patch#WithOwnedConditions), -and offers other options like [`patch.WithObservedGeneration`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/patch#WithStatusObservedGeneration). - -For all available functions and examples, see the [package reference](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/patch). - -### Runtime configuration options - -Several packages are available to align common runtime configuration flags across a set of controllers, easing the -end-user operator experience. - -| Package | Description | Reference | -|---|---|---| -| `client` | Kubernetes runtime client configurations like QPS and burst | [![GoDoc](https://pkg.go.dev/badge/github.com/fluxcd/pkg/runtime/client?utm_source=godoc)](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/client) | -| `leaderelection` | Kubernetes leader election configurations like the lease duration | [![GoDoc](https://pkg.go.dev/badge/github.com/fluxcd/pkg/runtime/leaderelection?utm_source=godoc)](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/leaderelection) | -| `logger` | Runtime logger configurations like the encoding and log level | [![GoDoc](https://pkg.go.dev/badge/github.com/fluxcd/pkg/runtime/logger?utm_source=godoc)](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/logger) | - ### Debugging The [`pprof`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/pprof) package allows setting up additional [Go `pprof`](https://golang.org/pkg/net/http/pprof/) From b6d8262744a3ce9b12bb534a998228c2a3eb6717 Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Mon, 12 Jul 2021 15:15:07 +0200 Subject: [PATCH 14/20] runtime: document dependency ordering in README Signed-off-by: Hidde Beydals --- runtime/README.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/runtime/README.md b/runtime/README.md index d0ba51e8a..38a825538 100644 --- a/runtime/README.md +++ b/runtime/README.md @@ -123,6 +123,16 @@ record metrics. | `controller.Events` | Provides the capabilities to send Events to the Kubernetes API and an external event recorder | [![GoDoc](https://pkg.go.dev/badge/github.com/fluxcd/pkg/runtime/controller?utm_source=godoc)](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/controller#Events) | | `controller.Metrics` | Provides the capabilities to record a set of common Prometheus metrics for a Kubernetes resource object | [![GoDoc](https://pkg.go.dev/badge/github.com/fluxcd/pkg/runtime/controller?utm_source=godoc)](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/controller#Metrics) | +### Dependency ordering + +The [`dependency`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/dependency) package provides a topological sort +function for Kubernetes resource objects that implement the [`dependency.Dependent`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/dependency#Dependent) +interface. For example used in situations where you want to be able to schedule reconciliations as +efficiently as possible, or in sequential batches in the right order. + +**NB:** at present the dependency ordering only works for dependencies of the same type, this should be changed or expanded in a +future iteration, partly because it would enable [flux2#1599](https://github.com/fluxcd/flux2/discussions/1599). + ### Debugging The [`pprof`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/pprof) package allows setting up additional [Go `pprof`](https://golang.org/pkg/net/http/pprof/) From f7a1d07175d69d9c4494f2515da4c155d3b70c8b Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Mon, 12 Jul 2021 15:25:14 +0200 Subject: [PATCH 15/20] runtime: note about conditions and meta API pkg Signed-off-by: Hidde Beydals --- runtime/README.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/runtime/README.md b/runtime/README.md index 38a825538..3a797659e 100644 --- a/runtime/README.md +++ b/runtime/README.md @@ -84,6 +84,10 @@ More specifically, it allows you to: For all available functions, see the [package reference](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/conditions). +The package combines well with the [genric meta API package types](https://pkg.go.dev/github.com/fluxcd/pkg/apis/meta), +and understands the [kstatus](#Supported-standards) Condition types if set up with +[`conditions.WithNegativePolarityConditions`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/conditions#WithNegativePolarityConditions). + ### Safe patching The [`patch`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/patch) package offers a helper utility to safely patch From fbaeedab43ca76f1650a7dcb63551ec2b60115a9 Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Mon, 12 Jul 2021 15:44:18 +0200 Subject: [PATCH 16/20] runtime: document errors in README Signed-off-by: Hidde Beydals --- runtime/README.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/runtime/README.md b/runtime/README.md index 3a797659e..e4a37f6b6 100644 --- a/runtime/README.md +++ b/runtime/README.md @@ -127,6 +127,12 @@ record metrics. | `controller.Events` | Provides the capabilities to send Events to the Kubernetes API and an external event recorder | [![GoDoc](https://pkg.go.dev/badge/github.com/fluxcd/pkg/runtime/controller?utm_source=godoc)](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/controller#Events) | | `controller.Metrics` | Provides the capabilities to record a set of common Prometheus metrics for a Kubernetes resource object | [![GoDoc](https://pkg.go.dev/badge/github.com/fluxcd/pkg/runtime/controller?utm_source=godoc)](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/controller#Metrics) | +### Errors + +The [`errors`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/errors) package contains a set of error types for +generic reconciler errors. This provides consistent error messages for end-users and allows for e.g. error type +checking. + ### Dependency ordering The [`dependency`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/dependency) package provides a topological sort From 5577a1bb131805fb51b87b45cf873832d2accb91 Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Mon, 12 Jul 2021 15:50:56 +0200 Subject: [PATCH 17/20] runtime: document predicates in README Signed-off-by: Hidde Beydals --- runtime/README.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/runtime/README.md b/runtime/README.md index e4a37f6b6..cf36a82c4 100644 --- a/runtime/README.md +++ b/runtime/README.md @@ -127,6 +127,16 @@ record metrics. | `controller.Events` | Provides the capabilities to send Events to the Kubernetes API and an external event recorder | [![GoDoc](https://pkg.go.dev/badge/github.com/fluxcd/pkg/runtime/controller?utm_source=godoc)](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/controller#Events) | | `controller.Metrics` | Provides the capabilities to record a set of common Prometheus metrics for a Kubernetes resource object | [![GoDoc](https://pkg.go.dev/badge/github.com/fluxcd/pkg/runtime/controller?utm_source=godoc)](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/controller#Metrics) | +### Predicates + +The [`predicates`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/predicates) package offers a set of [GitOps +Toolkit](https://fluxcd.io/docs/components/) common controller-runtime predicates, to for example notice annotation +changes. + +| Predicate | Description | Reference | +|---|---|---| +| `predicates.ReconcileRequestedPredicate` | Filter `meta.ReconcileRequestAnnotation` changes from update events | [![GoDoc](https://pkg.go.dev/badge/github.com/fluxcd/pkg/runtime/predicates?utm_source=godoc)](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/predicates#ReconcileRequestedPredicate) | + ### Errors The [`errors`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/errors) package contains a set of error types for From 883e09f52df9a44a10b080785771afc9624d22f9 Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Mon, 12 Jul 2021 16:22:09 +0200 Subject: [PATCH 18/20] runtime: list controller-runtime as "standard" Signed-off-by: Hidde Beydals --- runtime/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/runtime/README.md b/runtime/README.md index cf36a82c4..4e83f79f2 100644 --- a/runtime/README.md +++ b/runtime/README.md @@ -30,6 +30,7 @@ must be) used together to help with common operations. The packages build upon the following standards: +- [controller-runtime](https://github.com/kubernetes-sigs/controller-runtime) - [Kubernetes API conventions](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md) - [Kubernetes meta API conditions](https://github.com/kubernetes/enhancements/blob/master/keps/sig-api-machinery/1623-standardize-conditions/README.md) - [kstatus](https://github.com/kubernetes-sigs/cli-utils/tree/master/pkg/kstatus) From e22885e6f1ebf3edaf4101ff87ed9b261fecd3af Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Mon, 12 Jul 2021 16:27:14 +0200 Subject: [PATCH 19/20] runtime: link to envtest doc in README Signed-off-by: Hidde Beydals --- runtime/README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/runtime/README.md b/runtime/README.md index 4e83f79f2..5632f95c7 100644 --- a/runtime/README.md +++ b/runtime/README.md @@ -166,11 +166,11 @@ instructions on how to use the package. ### Testing The [`testenv`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/testenv) package can be utilized to control the -lifecycle of a local Kubernetes api-server used for testing purposes, and offers a set of helper utilities to -work with resources on the cluster. +lifecycle of a local Kubernetes api-server used for testing purposes, it is built upon controller-runtime's [`envtest`](https://book.kubebuilder.io/reference/envtest.html), +and can be further configured using the same environment variables. -It allows control over the runtime scheme and -Custom Resource Defintions using [`testenv.WithScheme`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/testenv#WithCRDPath) +The package offers an additional set of helper utilities around `envtest`, allowing configuration of the runtime +scheme and Custom Resource Definitions using [`testenv.WithScheme`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/testenv#WithCRDPath) and [`testenv.WithCRDPath`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/testenv#WithCRDPath). For all available functions, see the [package reference](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/testenv). From 4599158c1d62514fad767d75c5768e3fbbfea49c Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Mon, 12 Jul 2021 18:36:35 +0200 Subject: [PATCH 20/20] runtime: address review nits README Signed-off-by: Hidde Beydals --- runtime/README.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/runtime/README.md b/runtime/README.md index 5632f95c7..023d69882 100644 --- a/runtime/README.md +++ b/runtime/README.md @@ -24,7 +24,8 @@ must be) used together to help with common operations. - Become a toolbox for all problems, packages must be of interest to a wide range of controllers (and specifically, their runtime operations) before introduction should be considered. -- Adopt conflicting standards without breaking MAJOR version; opinions with versioning. +- Adopting every relevant standard. Potential integrations will be carefully considered, and where they break backward + compatibility, introduced only in major version releases. ## Supported standards @@ -120,7 +121,7 @@ The package is best used in combination with the [`controller.Events` helper](#C ### Controller helpers The [`controller`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/controller) package offers a collection of helpers -that can be embedded into a reconciler struct to provide them the capabilitiy to e.g. send Kubernetes Events and/or +that can be embedded into a reconciler struct to provide them the capability to e.g. send Kubernetes Events and/or record metrics. | Helper | Description | Reference | @@ -131,7 +132,7 @@ record metrics. ### Predicates The [`predicates`](https://pkg.go.dev/github.com/fluxcd/pkg/runtime/predicates) package offers a set of [GitOps -Toolkit](https://fluxcd.io/docs/components/) common controller-runtime predicates, to for example notice annotation +Toolkit](https://fluxcd.io/docs/components/) common controller-runtime predicates, for example to notice annotation changes. | Predicate | Description | Reference |