Skip to content

feat: allow using prometheus-client crate with PrometheusClientLayer#3134

Merged
Xuanwo merged 18 commits intoapache:mainfrom
flaneur2020:add-prometheus-client
Sep 19, 2023
Merged

feat: allow using prometheus-client crate with PrometheusClientLayer#3134
Xuanwo merged 18 commits intoapache:mainfrom
flaneur2020:add-prometheus-client

Conversation

@flaneur2020
Copy link
Copy Markdown
Contributor

@flaneur2020 flaneur2020 commented Sep 19, 2023

I'm currently working on switching to prometheus from metrics-rs in databend at databendlabs/databend#12787 , but unfortunately the PrometheusLayer in opendal uses the tikv/prometheus-rs crate, but databend uses prometheus-client from the prometheus official.

This PR addes a feature option use-prometheus-client to allow switching to prometheus-client with the PrometheusLayer. When use-prometheus-client is enabled, we can pass a prometheus_client::registry::Registry to a PrometheusLayer.

To support both of the prometheus client, this PR also make a small refactor to collect the metrics writing logics into a trait PrometheusLayerMetrics.

I believe the official prometheus may have longer term of support, maybe we can have a schedule to migrate to it while deprecate the older prometheus library's support?

@github-actions github-actions Bot added the releases-note/feat The PR implements a new feature or has a title that begins with "feat" label Sep 19, 2023
@flaneur2020 flaneur2020 marked this pull request as draft September 19, 2023 07:11
@flaneur2020 flaneur2020 marked this pull request as ready for review September 19, 2023 07:13
@Xuanwo
Copy link
Copy Markdown
Member

Xuanwo commented Sep 19, 2023

I would add a new layer called PrometheusClientLayer instead of maintaining two different features. Playing with features is not fun 😢

@flaneur2020
Copy link
Copy Markdown
Contributor Author

@Xuanwo you're right, what a pain. I've refactored to another PrometheusClientLayer, PTAL

@flaneur2020 flaneur2020 changed the title feat: allow using prometheus-client crate with PrometheusLayer feat: allow using prometheus-client crate with PrometheusClientLayer Sep 19, 2023
Comment thread core/src/layers/prometheus_client.rs
Comment thread core/src/layers/prometheus_client.rs Outdated
Comment thread core/src/layers/prometheus_client.rs Outdated
Comment thread core/src/layers/prometheus_client.rs Outdated
Comment thread core/src/layers/prometheus_client.rs Outdated
Comment thread core/src/layers/prometheus_client.rs Outdated
Comment thread core/src/layers/prometheus_client.rs Outdated
Comment thread core/src/layers/prometheus_client.rs Outdated
Comment thread core/src/layers/prometheus_client.rs Outdated
Comment thread core/src/layers/prometheus_client.rs
@flaneur2020 flaneur2020 requested a review from Xuanwo September 19, 2023 11:11
Copy link
Copy Markdown
Member

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, let's go!

@Xuanwo
Copy link
Copy Markdown
Member

Xuanwo commented Sep 19, 2023

The only thing left is to make our CI happy.

@Xuanwo Xuanwo merged commit 30de95e into apache:main Sep 19, 2023
Young-Flash pushed a commit to Young-Flash/opendal that referenced this pull request Sep 19, 2023
apache#3134)

* refactor prometheus layer

* add prometheus-client to deps

* chore: simplify imports

* refactor the metrics into a trait

* feat: add implementation with prometheus-client

* fix: allow using different trait

* cargo fmt

* refactor: add a seperate layer

* fix: docs

* fix typo

* fix: cargo fmt

* add a prefix

* use structed labels

* use labels in array

* remove the unused metrics

* rename stats to metrics

* record request duration in wrapper

* fix fmt
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

releases-note/feat The PR implements a new feature or has a title that begins with "feat"

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants