TRACING-5814 | RHOSDT 3.9 Document google managed prometheus

[pavolloffay]

Version(s):

RHOSDT 3.9
OCP 4.12-4.21 (all supported OCP versions at the merge time)

Issue:

https://issues.redhat.com/browse/TRACING-5814

Link to docs preview:

QE review:

• QE has approved this change.

Additional information:

• GMP OTLP metrics ingestion user facing doc https://docs.google.com/document/d/1FCHI6Qwv5F522j2ajd6Nhoq8FRTPutoVYDMYgZqagcw/edit?tab=t.0
• https://docs.google.com/document/d/1-C2Fx9BV58t7iyEQhWItBzO_3vC3HAM0_5hEfYLCM4o/edit?tab=t.0
• sample https://github.com/GoogleCloudPlatform/opentelemetry-operator-sample/blob/main/recipes/host-and-kubelet-metrics/collector-config.yaml

OTELcol 0.142.0

https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CHANGELOG.md#-breaking-changes-

receiver/prometheus: Promote the receiver.prometheusreceiver.RemoveStartTimeAdjustment feature gate to stable and remove in-receiver metric start time adjustment in favor of the metricstarttime processor, including disabling the created-metric feature gate. (#44180) Previously, users could disable the RemoveStartTimeAdjustment feature gate to temporarily keep the legacy start time adjustment behavior in the Prometheus receiver. With this promotion to stable and bounded registration, that gate can no longer be disabled; the receiver will no longer set StartTime on metrics based on process_start_time_seconds, and users should migrate to the metricstarttime processor for equivalent functionality. This change also disables the receiver.prometheusreceiver.UseCreatedMetric feature gate, which previously used the <metric>_created series to derive start timestamps for counters, summaries, and histograms when scraping non OpenMetrics protocols. However, this does not mean that the _created series is always ignored: when using the OpenMetrics 1.0 protocol, Prometheus itself continues to interpret the _created series as the start timestamp, so only the receiver-side handling for other scrape protocols has been removed.

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[pavolloffay]

/test all

[ocpdocs-previewbot]

🤖 Thu Jan 08 12:25:44 - Prow CI generated the docs preview:

https://104286--ocpdocs-pr.netlify.app/openshift-enterprise/latest/observability/otel/otel-collector/otel-collector-extensions.html
https://104286--ocpdocs-pr.netlify.app/openshift-enterprise/latest/observability/otel/otel-forwarding-telemetry-data.html

[IshwarKanse]

@pavolloffay Should we also document the authentication bits using GCP SA and WIF ?
https://github.com/openshift/distributed-tracing-qe/blob/main/tests/e2e-otel/googlemanagedprometheus/gcp-wif/otel-wif-collector.yaml
https://github.com/openshift/distributed-tracing-qe/blob/main/tests/e2e-otel/googlemanagedprometheus/gcp-sa/otel-sa-collector.yaml

[pavolloffay]

/test all

[pavolloffay]

@IshwarKanse the enxtension has a project parameter

project - The Google Cloud Project telemetry is sent to if the gcp.project.id resource attribute is not set. If unspecified, this is determined using application default credentials.

https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/extension/googleclientauthextension

If unspecified, this is determined using application default credentials.

what do they mean by this? In OCP context, can it be only specified in GOOGLE_APPLICATION_CREDENTIALS?

[IshwarKanse]

The Application Default Credentials (ADC) is Google's credential discovery mechanism that searches for credentials in this order:

1. GOOGLE_APPLICATION_CREDENTIALS environment variable (pointing to a credential JSON file)
2. Local ADC file at ~/.config/gcloud/application_default_credentials.json (created by gcloud auth application-default login)
3. GCP metadata server (for workloads running on Google Cloud infrastructure)

In the OpenShift/Kubernetes context, GOOGLE_APPLICATION_CREDENTIALS is the standard approach, even when running on Google Cloud, because:

• It provides explicit control over which credentials the collector uses
• It's portable across different environments (on-prem, AWS, Azure, GCP, etc.)
• The local ADC file option (Added client tools install topics #2) is meant for local development, not production containers
• While the metadata server (Initial build system #3) would work for OpenShift clusters running on GCP, explicitly mounting credentials is the Kubernetes best practice

The GOOGLE_APPLICATION_CREDENTIALS env var should point to one of these credential types:

1. Service Account Key JSON:
   env:
   - name: GOOGLE_APPLICATION_CREDENTIALS
   value: "/etc/google-cloud-sa/sa-key.json"
2. Workload Identity Federation config JSON:
   env:
   - name: GOOGLE_APPLICATION_CREDENTIALS
   value: "/etc/workload-identity/credential-configuration.json"

About the project parameter

When we set project in the extension config, it's actually optional because:

• ADC will automatically extract the project ID from the credential file
• It only serves as a fallback when the gcp.project.id resource attribute isn't set
• In my referenced config in the previous comment, we are already setting gcp.project_id via the resource processor, so the extension's project parameter is somewhat redundant

For the documentation, I'd suggest making the project parameter optional and explaining that users can omit it if they're already setting the resource attribute or if they want ADC to auto-detect it from the credentials.

[pavolloffay]

/test all

[pavolloffay]

/test all

[openshift-ci]

@pavolloffay: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.