-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Create admin guide #3526
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Create admin guide #3526
Changes from all commits
ae3afb0
fbf49f0
22868bc
46c2d14
2153507
0f46f58
9d3093f
4af5630
cc6153d
1513812
a130553
a9b89f2
543e6fa
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| --- | ||
| title: "Administration guide" | ||
| weight: 10 | ||
| type: "docs" | ||
| showlandingtoc: "true" | ||
| --- | ||
|
|
||
| There are two core Knative components that can be installed and used together or independently to provide different functions: | ||
|
|
||
| * [Knative Serving](../serving/): Easily manage stateless services on Kubernetes by reducing the developer effort required for autoscaling, networking, and rollouts. | ||
|
|
||
| * [Knative Eventing](../eventing/): Easily route events between on-cluster and off-cluster components by exposing event routing as configuration rather than embedded in code. | ||
|
|
||
| These components are delivered as Kubernetes custom resource definitions (CRDs), which can be configured by a cluster administrator to provide default settings for developer-created applications and event workflow components. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,176 @@ | ||
| --- | ||
| title: "Logging" | ||
| weight: 50 | ||
| type: "docs" | ||
| aliases: | ||
| - /docs/install/collecting-logs | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we want to make "logs" a top-level next to "install", add a "configuration" sub-tree to hold all the configuration, or have a "common" sub-tree to hold configuration that's common to Serving and Eventing (off the top of my head, the high-availability configuration, logging, and monitoring)? (I'm trying to avoid the "too many items at one level in the LHS navigation" problem we have today.)
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can you add this as a suggestion to the Miro doc please so I can see better what you're saying / how this fits in?
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @evankanderson tracing too? |
||
| showlandingtoc: "false" | ||
| --- | ||
|
|
||
| You can use [Fluent Bit](https://docs.fluentbit.io/), a log processor and forwarder, to collect your kubernetes logs in a central directory. | ||
|
|
||
| This is not required to run Knative, but can be helpful with [Knative Serving](../serving), which will automatically delete pods and associated logs when they are no longer needed. | ||
|
|
||
| Fluent Bit supports exporting to a number of other log providers. If you already have an existing log provider, for example, Splunk, Datadog, ElasticSearch, or Stackdriver, you can follow the [FluentBit documentation](https://docs.fluentbit.io/manual/pipeline/outputs) to configure log forwarders. | ||
|
|
||
| Setting up log collection requires two steps: | ||
|
|
||
| 1. Running a log forwarding DaemonSet on each node. | ||
| 2. Running a collector somewhere in the cluster. | ||
|
|
||
| In the following example, a StatefulSet is used, which stores logs on a Kubernetes PersistentVolumeClaim, but you can also use a HostPath. | ||
|
|
||
| ## Setting up the collector | ||
|
|
||
| Set up the collector before the forwarders. You will need the address of the collector when configuring the forwarders, and the forwarders may queue logs until the collector is ready. | ||
|
|
||
|  | ||
|
|
||
| <!-- yuml.me UML rendering of: | ||
| [Forwarder1]logs->[Collector] | ||
| [Forwarder2]logs->[Collector] | ||
|
|
||
| // Add notes | ||
| [Collector]->[shared volume] | ||
| [nginx]-[shared volume] | ||
| --> | ||
|
|
||
| The [`fluent-bit-collector.yaml`](./fluent-bit-collector.yaml) defines a | ||
| StatefulSet, as well as a Kubernetes Service which allows accessing and reading | ||
| the logs from within the cluster. The supplied configuration will create the | ||
| monitoring configuration in a namespace called `logging`. | ||
|
|
||
| You can apply the configuration by entering the command: | ||
|
|
||
| ```shell | ||
| kubectl apply --filename https://github.com/knative/docs/raw/main/docs/install/collecting-logs/fluent-bit-collector.yaml | ||
| ``` | ||
|
|
||
| The default configuration will classify logs into: | ||
|
|
||
| - Knative apps, or pods with an `app=Knative` label | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should apps be "services" here?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think it's the same difference but I didn't write this, I'd imagine @skonto did maybe, I just cleaned it up, so let's get his opinion before we change any wording relating to components?
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I didnt write this part but I think using |
||
| - Non-Knative apps | ||
|
|
||
| The logs default to logging with the pod name; this can be changed by updating the `log-collector-config` ConfigMap before or after installation. Once the ConfigMap is updated, you must restart Fluent Bit. You can do this by deleting the pod and letting the StatefulSet recreate it. | ||
|
|
||
| To access the logs through your web browser, enter the command: | ||
|
|
||
| ```shell | ||
| kubectl port-forward --namespace logging service/log-collector 8080:80 | ||
| ``` | ||
|
|
||
| And then visit http://localhost:8080/. | ||
|
|
||
| You can also open a shell in the `nginx` pod and search the logs using unix | ||
| tools, by entering the command: | ||
|
|
||
| ```shell | ||
| kubectl exec --namespace logging --stdin --tty --container nginx log-collector-0 | ||
| ``` | ||
|
|
||
| ## Setting up the forwarders | ||
|
|
||
| Follow the [Fluent Bit directions for installing on Kubernetes](https://docs.fluentbit.io/manual/installation/kubernetes) documentation to set up a Fluent Bit DaemonSet which forwards logs to ElasticSearch by default. | ||
|
|
||
| When the directions call for creating the ConfigMap, you need to replace the ElasticSearch configuration with the [`fluent-bit-configmap.yaml`](./fluent-bit-configmap.yaml), or add the following block to the ConfigMap, and update the | ||
| `@INCLUDE output-elasticsearch.conf` to be `@INCLUDE output-forward.conf`. | ||
|
|
||
| ```yaml | ||
| output-forward.conf: | | ||
| [OUTPUT] | ||
| Name forward | ||
| Host log-collector.logging | ||
| Port 24224 | ||
| Require_ack_response True | ||
| ``` | ||
|
|
||
| ## Local collector | ||
|
|
||
| **NOTE:** This describes a development environment setup, and is not appropriate | ||
|
abrennan89 marked this conversation as resolved.
|
||
| for production. | ||
|
|
||
| If you are using a local Kubernetes cluster for development, you can create a `hostPath` PersistentVolume to store the logs on your desktop operating system. This allows you to use your usual desktop tools on the files without needing Kubernetes-specific tools. | ||
|
|
||
| The PersistentVolumeClaim will look something like this, but the `hostPath` will | ||
| vary based on your Kubernetes software and host operating system. Some example | ||
| values are documented below. | ||
|
|
||
| ```yaml | ||
|
abrennan89 marked this conversation as resolved.
|
||
| apiVersion: v1 | ||
| kind: PersistentVolume | ||
| metadata: | ||
| name: shared-logs | ||
| labels: | ||
| app: logs-collector | ||
| spec: | ||
| accessModes: | ||
| - "ReadWriteOnce" | ||
| storageClassName: manual | ||
| claimRef: | ||
| apiVersion: v1 | ||
| kind: PersistentVolumeClaim | ||
| name: logs-log-collector-0 | ||
| namespace: logging | ||
| capacity: | ||
| storage: 5Gi | ||
| hostPath: | ||
| path: <see below> | ||
| ``` | ||
|
|
||
| You must update the StatefulSet `volumeClaimTemplates` to reference the `shared-logs` volume, as shown in the following example: | ||
|
|
||
| ```yaml | ||
| volumeClaimTemplates: | ||
| metadata: | ||
| name: logs | ||
| spec: | ||
| accessModes: ["ReadWriteOnce"] | ||
| volumeName: shared-logs | ||
| ``` | ||
|
|
||
| ### Kind | ||
|
|
||
| When creating your cluster, you must use a `kind-config.yaml` and specify | ||
| `extraMounts` for each node, as shown in the following example: | ||
|
|
||
| ```yaml | ||
| apiversion: kind.x-k8s.io/v1alpha4 | ||
| kind: Cluster | ||
| nodes: | ||
| - role: control-plane | ||
| extraMounts: | ||
| - hostPath: ./logs | ||
| containerPath: /shared/logs | ||
| - role: worker | ||
| extraMounts: | ||
| - hostPath: ./logs | ||
| containerPath: /shared/logs | ||
| ``` | ||
|
|
||
| You can then use `/shared/logs` as the `spec.hostPath.path` in your | ||
| PersistentVolume. Note that the directory path `./logs` is relative to the | ||
| directory that the Kind cluster was created in. | ||
|
|
||
| ### Docker Desktop | ||
|
|
||
| Docker desktop automatically creates some shared mounts between the host and the | ||
| guest operating systems, so you only need to know the path to your home | ||
| directory. The following are some examples for different operating systems: | ||
|
|
||
| | Host OS | `hostPath` | | ||
| | ------- | ---------------------------------------- | | ||
| | Mac OS | `/Users/${USER}` | | ||
| | Windows | `/run/desktop/mnt/host/c/Users/${USER}/` | | ||
| | Linux | `/home/${USER}` | | ||
|
|
||
| ### Minikube | ||
|
|
||
| Minikube requires an explicit command to [mount a directory](https://minikube.sigs.k8s.io/docs/handbook/mount/) into the virtual machine (VM) running Kubernetes. | ||
|
|
||
| The following command mounts the `logs` directory inside the current directory onto `/mnt/logs` in the VM: | ||
|
|
||
| ```shell | ||
| minikube mount ./logs:/mnt/logs | ||
| ``` | ||
|
|
||
| You must also reference `/mnt/logs` as the `hostPath.path` in the PersistentVolume. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think there's also some common configuration options for administrators around telemetry (
config-logging,config-observability) that should probably move to the admin guide:https://knative.dev/docs/install/collecting-metrics/
https://knative.dev/docs/install/collecting-logs/
(I'm not quite sure what to do with the "here's a local-logs-collection solution" at the end of the second; probably turn it into something real or go back and delete it.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nevermind, it looks like you did this. Do you want to mention "common configuration options" as another section here, or just rely on the left-hand nav?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't want to add a bunch of links to maintain really until things settle down a bit with moving stuff, but for now I've added the contents list back to the page; does that work for you?