diff --git a/_topic_map.yml b/_topic_map.yml index 14d1107a8813..366bb8db2e75 100644 --- a/_topic_map.yml +++ b/_topic_map.yml @@ -988,6 +988,8 @@ Topics: File: examining-cluster-metrics - Name: Accessing Prometheus, Alertmanager, and Grafana File: prometheus-alertmanager-and-grafana +- Name: Monitoring your own services + File: monitoring-your-own-services - Name: Exposing custom application metrics for autoscaling File: exposing-custom-application-metrics-for-autoscaling --- diff --git a/images/monitoring-metrics-developer.png b/images/monitoring-metrics-developer.png new file mode 100644 index 000000000000..cbd0abaa88c0 Binary files /dev/null and b/images/monitoring-metrics-developer.png differ diff --git a/modules/monitoring-accessing-the-metrics-of-your-service.adoc b/modules/monitoring-accessing-the-metrics-of-your-service.adoc new file mode 100644 index 000000000000..2e8e0baafbda --- /dev/null +++ b/modules/monitoring-accessing-the-metrics-of-your-service.adoc @@ -0,0 +1,43 @@ +// Module included in the following assemblies: +// +// * monitoring/monitoring-your-own-services.adoc + +[id="accessing-the-metrics-of-your-service_{context}"] += Accessing the metrics of your service + +Once you have enabled monitoring your own services, deployed a service, and set up metrics collection for it, you can access the metrics of the service as a cluster administrator, as a developer, or as a user with view permissions for the project. + +.Prerequisites + +* You need to deploy the service that you want to monitor. +* You need to enable monitoring of your own services. +* You need to have metrics scraping set up for the service. +* You need to log in as a cluster administrator, a developer, or as a user with view permissions for the project. + +.Procedure + +. Access the Prometheus web interface: ++ +* To access the metrics as a cluster administrator, go to the {product-title} web console, switch to the Administrator Perspective, and click *Monitoring* -> *Metrics*. ++ +[NOTE] +==== +Cluster administrators, when using the Administrator Perspective, have access to all cluster metrics and to custom service metrics from all projects. +==== ++ +[NOTE] +==== +Only cluster administrators have access to the Alertmanager and Prometheus UIs. +==== ++ +* To access the metrics as a developer or a user with permissions, go to the {product-title} web console, switch to the Developer Perspective, then click *Advanced* -> *Metrics*. Select the project you want to see the metrics for. ++ +[NOTE] +==== +Developers can only use the Developer Perspective. They can only query metrics from a single project. +==== +. Use the PromQL interface to run queries for your services. + +.Additional resources + +* See the xref:../monitoring/cluster-monitoring/examining-cluster-metrics.adoc#examining-cluster-metrics[section on using the PromQL interface]. diff --git a/modules/monitoring-creating-a-role-for-setting-up-metrics-collection.adoc b/modules/monitoring-creating-a-role-for-setting-up-metrics-collection.adoc new file mode 100644 index 000000000000..af2a54cb8824 --- /dev/null +++ b/modules/monitoring-creating-a-role-for-setting-up-metrics-collection.adoc @@ -0,0 +1,36 @@ +// Module included in the following assemblies: +// +// * monitoring/monitoring-your-own-services.adoc + +[id="creating-a-role-for-setting-up-metrics-collection_{context}"] += Creating a role for setting up metrics collection + +This procedure shows how to create a role that allows a user to set up metrics collection for a service as described in "Setting up metrics collection". + +.Procedure + +. Create a YAML file for the new role. In this example, it is called `custom-metrics-role.yaml`. + +. Fill the file with the configuration for the `monitor-crd-edit` role: ++ +[source,yaml] +---- +kind: ClusterRole +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: monitor-crd-edit +rules: +- apiGroups: ["monitoring.coreos.com"] + resources: ["prometheusrules", "servicemonitors", "podmonitors"] + verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] +---- ++ +This role enables a user to set up metrics collection for services. + +. Apply the configuration file to the cluster: ++ +---- +$ oc apply -f custom-metrics-role.yaml +---- ++ +Now the role is created. diff --git a/modules/monitoring-creating-alerting-rules.adoc b/modules/monitoring-creating-alerting-rules.adoc new file mode 100644 index 000000000000..541fbb71e75a --- /dev/null +++ b/modules/monitoring-creating-alerting-rules.adoc @@ -0,0 +1,49 @@ +// Module included in the following assemblies: +// +// * monitoring/monitoring-your-own-services.adoc + +[id="creating-alerting-rules_{context}"] += Creating alerting rules + +You can create alerting rules, which will fire alerts based on values of chosen metrics of the service. + +[NOTE] +==== +In the current version of the Technology Preview, only administrators can access alerting rules using the Prometheus UI and the Web Console. +==== + +.Procedure + +. Create a YAML file for alerting rules. In this example, it is called `example-app-alerting-rule.yaml`. + +. Fill the file with the configuration for the alerting rules: ++ +[NOTE] +==== +The expression can only reference metrics exposed by your own services. Currently it is not possible to correlate existing cluster metrics. +==== ++ +[source,yaml] +---- +apiVersion: monitoring.coreos.com/v1 +kind: PrometheusRule +metadata: + name: example-alert + namespace: ns1 +spec: + groups: + - name: example + rules: + - alert: VersionAlert + expr: version{job="prometheus-example-app"} == 0 +---- ++ +This configuration creates an alerting rule named `example-alert`, which fires an alert when the `version` metric exposed by the sample service becomes `0`. + +. Apply the configuration file to the cluster: ++ +---- +$ oc apply -f example-app-alerting-rule.yaml +---- ++ +It will take some time to create the alerting rules. diff --git a/modules/monitoring-deploying-a-sample-service.adoc b/modules/monitoring-deploying-a-sample-service.adoc new file mode 100644 index 000000000000..346b7396d8e0 --- /dev/null +++ b/modules/monitoring-deploying-a-sample-service.adoc @@ -0,0 +1,79 @@ +// Module included in the following assemblies: +// +// * monitoring/monitoring-your-own-services.adoc + +[id="deploying-a-sample-service_{context}"] += Deploying a sample service + +To test monitoring your own services, you can deploy a sample service. + +.Procedure + +. Create a YAML file for the service configuration. In this example, it is called `prometheus-example-app.yaml`. + +. Fill the file with the configuration for deploying the service: ++ +[source,yaml] +---- +apiVersion: v1 +kind: Namespace +metadata: + name: ns1 +--- +apiVersion: extensions/v1beta1 +kind: Deployment +metadata: + labels: + app: prometheus-example-app + name: prometheus-example-app + namespace: ns1 +spec: + replicas: 1 + selector: + matchLabels: + app: prometheus-example-app + template: + metadata: + labels: + app: prometheus-example-app + spec: + containers: + - image: quay.io/brancz/prometheus-example-app:v0.1.0 + imagePullPolicy: IfNotPresent + name: prometheus-example-app +--- +apiVersion: v1 +kind: Service +metadata: + labels: + app: prometheus-example-app + name: prometheus-example-app + namespace: ns1 +spec: + ports: + - port: 8080 + protocol: TCP + targetPort: 8080 + name: web + selector: + app: prometheus-example-app + type: ClusterIP +---- ++ +This configuration deploys a service named `prometheus-example-app` in the `ns1` project. This service exposes the custom `version` metric. + +. Apply the configuration file to the cluster: ++ +---- +$ oc apply -f prometheus-example-app.yaml +---- ++ +It will take some time to deploy the service. + +. You can check that the service is running: ++ +---- +$ oc -n ns1 get pod +NAME READY STATUS RESTARTS AGE +prometheus-example-app-7857545cb7-sbgwq 1/1 Running 0 81m +---- diff --git a/modules/monitoring-enabling-monitoring-of-your-own-services.adoc b/modules/monitoring-enabling-monitoring-of-your-own-services.adoc new file mode 100644 index 000000000000..479166225716 --- /dev/null +++ b/modules/monitoring-enabling-monitoring-of-your-own-services.adoc @@ -0,0 +1,50 @@ +// Module included in the following assemblies: +// +// * monitoring/monitoring-your-own-services.adoc + +[id="enabling-monitoring-of-your-own-services_{context}"] += Enabling monitoring of your own services + +You can enable monitoring your own services by setting the `techPreviewUserWorkload/enabled` flag in the cluster monitoring ConfigMap. + +.Prerequisites + +* Make sure you have the `cluster-monitoring-config` ConfigMap object with the `data/config.yaml` section. + +.Procedure + +. Start editing the `cluster-monitoring-config` ConfigMap: ++ +---- +$ oc -n openshift-monitoring edit configmap cluster-monitoring-config +---- + +. Set the `techPreviewUserWorkload` setting to `true` under `data/config.yaml`: ++ +---- +apiVersion: v1 +kind: ConfigMap +metadata: + name: cluster-monitoring-config + namespace: openshift-monitoring +data: + config.yaml: | + techPreviewUserWorkload: + enabled: true +---- + +. Save the file to apply the changes. Monitoring your own services is enabled automatically. + +. Optionally, you can check that the `prometheus-user-workload` pods were created: ++ +---- +$ oc -n openshift-user-workload-monitoring get pod +NAME READY STATUS RESTARTS AGE +prometheus-operator-85bbb7b64d-7jwjd 1/1 Running 0 3m24s +prometheus-user-workload-0 5/5 Running 1 3m13s +prometheus-user-workload-1 5/5 Running 1 3m13s +---- + +.Additional resources + +* See xref:../monitoring/cluster-monitoring/configuring-the-monitoring-stack.adoc#configuring-the-monitoring-stack[Configuring the monitoring stack] to learn how to create `cluster-monitoring-config`. diff --git a/modules/monitoring-examining-metrics-as-a-developer.adoc b/modules/monitoring-examining-metrics-as-a-developer.adoc new file mode 100644 index 000000000000..70957bbb84a9 --- /dev/null +++ b/modules/monitoring-examining-metrics-as-a-developer.adoc @@ -0,0 +1,12 @@ +// Module included in the following assemblies: +// +// * monitoring/cluster-monitoring/examining-cluster-metrics.adoc + +[id="non-administrator-access-to-metrics_{context}"] += Non-administrator access to metrics + +As a developer, one can enable user workload monitoring for an application or service in a project. As an administrator, you use the same feature to enable monitoring for infrastructure workloads. In that case, a developer or administrator of that project can examine the exposed metrics using the Developer Perspective in the Web console. + +:FeatureName: Examining metrics using the Developer Perspective +include::modules/technology-preview.adoc[leveloffset=+0] + diff --git a/modules/monitoring-giving-view-access-to-a-user.adoc b/modules/monitoring-giving-view-access-to-a-user.adoc new file mode 100644 index 000000000000..1e6406e2b71f --- /dev/null +++ b/modules/monitoring-giving-view-access-to-a-user.adoc @@ -0,0 +1,29 @@ +// Module included in the following assemblies: +// +// * monitoring/monitoring-your-own-services.adoc + +[id="giving-view-access-to-a-user_{context}"] += Giving view access to a user + +By default, only cluster administrator users and developers have access to metrics from your services. This procedure shows how to grant metrics access to a particular project to an arbitrary user. + +.Prerequisites + +* You need to have a user created. +* You need to log in as a cluster administrator. + +.Procedure + +* Run this command to give access to all metrics of your services in : ++ +---- +$ oc policy add-role-to-user view -n +---- ++ +For example, to give view access to the `ns1` namespace to user `bobwilliams`, run: ++ +---- +$ oc policy add-role-to-user view bobwilliams -n ns1 +---- + +* Alternatively, in the Web console, switch to the Developer Perspective, and click *Advanced* -> *Project Access*. From there, you can select the correct namespace and assign the `view` role to a user. diff --git a/modules/monitoring-granting-the-role-to-a-user.adoc b/modules/monitoring-granting-the-role-to-a-user.adoc new file mode 100644 index 000000000000..b6a8b130d6be --- /dev/null +++ b/modules/monitoring-granting-the-role-to-a-user.adoc @@ -0,0 +1,24 @@ +// Module included in the following assemblies: +// +// * monitoring/monitoring-your-own-services.adoc + +[id="granting-the-role-to-a-user_{context}"] += Granting the role to a user + +This procedure shows how to assign the `monitor-crd-edit` role to a user. + +.Prerequisites + +* You need to have a user created. +* You need to have the `monitor-crd-edit` role described in "Creating a role for setting up metrics collection" created. + +.Procedure + +. In the Web console, navigate to *User Management* -> *Role Bindings* -> *Create Binding*. +. In *Binding Type*, select the "Namespace Role Binding" type. +. In *Name*, enter a name for the binding. +. In *Namespace*, select the namespace where you want to grant the access. +. In *Role Name*, enter `monitor-crd-edit`. +. In *Subject*, select *User*. +. In *Subject Name*, enter name of the user, for example `johnsmith`. +. Confirm the role binding. Now the user has been assigned the `monitor-crd-edit` role, which allows him to set up metrics collection for a service in the namespace. diff --git a/modules/monitoring-setting-up-metrics-collection.adoc b/modules/monitoring-setting-up-metrics-collection.adoc new file mode 100644 index 000000000000..7691381ff667 --- /dev/null +++ b/modules/monitoring-setting-up-metrics-collection.adoc @@ -0,0 +1,61 @@ +// Module included in the following assemblies: +// +// * monitoring/monitoring-your-own-services.adoc + +[id="setting-up-metrics-collection_{context}"] += Setting up metrics collection + +To use the metrics exposed by your service, you need to configure OpenShift Monitoring to scrape metrics from the `/metrics` endpoint. You can do this using a ServiceMonitor, a custom resource definition (CRD) that specifies how a service should be monitored, or a PodMonitor, a CRD that specifies how a pod should be monitored. The former requires a Service object, while the latter does not, allowing Prometheus to directly scrape metrics from the metrics endpoint exposed by a Pod. + +This procedure shows how to create a ServiceMonitor for the service. + +.Prerequisites + +* Log in as a cluster administrator or a user with the `monitor-crd-edit` role. + +.Procedure + +. Create a YAML file for the ServiceMonitor configuration. In this example, the file is called `example-app-service-monitor.yaml`. + +. Fill the file with the configuration for creating the ServiceMonitor: ++ +[source,yaml] +---- +apiVersion: monitoring.coreos.com/v1 +kind: ServiceMonitor +metadata: + labels: + k8s-app: prometheus-example-monitor + name: prometheus-example-monitor + namespace: ns1 +spec: + endpoints: + - interval: 30s + port: web + scheme: http + selector: + matchLabels: + app: prometheus-example-app +---- ++ +This configuration makes OpenShift Monitoring scrape the metrics exposed by the sample service deployed in "Deploying a sample service", which includes the single `version` metric. + +. Apply the configuration file to the cluster: ++ +---- +$ oc apply -f example-app-service-monitor.yaml +---- ++ +It will take some time to deploy the ServiceMonitor. + +. You can check that the ServiceMonitor is running: ++ +---- +$ oc -n ns1 get servicemonitor +NAME AGE +prometheus-example-monitor 81m +---- + +.Additional resources + +See the link:https://github.com/openshift/prometheus-operator/blob/release-4.3/Documentation/api.md[Prometheus Operator API documentation] for more information on ServiceMonitors and PodMonitors. diff --git a/monitoring/cluster-monitoring/examining-cluster-metrics.adoc b/monitoring/cluster-monitoring/examining-cluster-metrics.adoc index 9fe6461bd17b..a11678364b4e 100644 --- a/monitoring/cluster-monitoring/examining-cluster-metrics.adoc +++ b/monitoring/cluster-monitoring/examining-cluster-metrics.adoc @@ -10,6 +10,11 @@ toc::[] include::modules/monitoring-contents-of-the-metrics-ui.adoc[leveloffset=+1] include::modules/monitoring-running-metrics-queries.adoc[leveloffset=+1] include::modules/monitoring-exploring-the-visualized-metrics.adoc[leveloffset=+1] +include::modules/monitoring-examining-metrics-as-a-developer.adoc[leveloffset=+1] + +.Additional resources + +See the xref:../../monitoring/monitoring-your-own-services.adoc#monitoring-your-own-services[documentation on monitoring your own services]. It includes details on accessing non-cluster metrics as a developer or a privileged user. .Next steps diff --git a/monitoring/monitoring-your-own-services.adoc b/monitoring/monitoring-your-own-services.adoc new file mode 100644 index 000000000000..433a70cdb209 --- /dev/null +++ b/monitoring/monitoring-your-own-services.adoc @@ -0,0 +1,26 @@ +[id="monitoring-your-own-services"] += Monitoring your own services +include::modules/common-attributes.adoc[] +:context: monitoring-your-own-services + +toc::[] + +You can use OpenShift Monitoring for your own services in addition to monitoring the cluster. This way, you do not need to use an additional monitoring solution. This helps keeping monitoring centralized. +Additionally, you can extend the access to the metrics of your services beyond cluster administrators. This enables developers and arbitrary users to access these metrics. + +[NOTE] +==== +Opting into monitoring your own services is mutually exclusive with either a custom installation of Prometheus Operator or installing Prometheus Operator using Operator Lifecycle Manager (OLM). +==== + +:FeatureName: Monitoring your own services +include::modules/technology-preview.adoc[leveloffset=+0] + +include::modules/monitoring-enabling-monitoring-of-your-own-services.adoc[leveloffset=+1] +include::modules/monitoring-deploying-a-sample-service.adoc[leveloffset=+1] +include::modules/monitoring-creating-a-role-for-setting-up-metrics-collection.adoc[leveloffset=+1] +include::modules/monitoring-granting-the-role-to-a-user.adoc[leveloffset=+1] +include::modules/monitoring-setting-up-metrics-collection.adoc[leveloffset=+1] +include::modules/monitoring-creating-alerting-rules.adoc[leveloffset=+1] +include::modules/monitoring-giving-view-access-to-a-user.adoc[leveloffset=+1] +include::modules/monitoring-accessing-the-metrics-of-your-service.adoc[leveloffset=+1]