Skip to content

refactor(layers/prometheus): provide builder APIs#5072

Merged
Xuanwo merged 19 commits intoapache:mainfrom
koushiro-contrib:prometheus-consistent-apis
Sep 3, 2024
Merged

refactor(layers/prometheus): provide builder APIs#5072
Xuanwo merged 19 commits intoapache:mainfrom
koushiro-contrib:prometheus-consistent-apis

Conversation

@koushiro
Copy link
Copy Markdown
Member

@koushiro koushiro commented Aug 30, 2024

Which issue does this PR close?

Related issue: #5069 (comment)
Related PR: #5073

Rationale for this change

Provide similar APIs for PrometheusLayer and PrometheusClientLayer

What changes are included in this PR?

  • add PrometheusLayer::builder
  • add PrometheusLayerBuilder
  • impl Default for PrometheusLayer
  • move path_label_value into observe mod
  • add more examples in doc

Are there any user-facing changes?

API breaking change

@koushiro koushiro requested a review from Xuanwo as a code owner August 30, 2024 05:13
@github-actions github-actions Bot added the releases-note/refactor The PR does a refactor on code or has a title that begins with "refactor" label Aug 30, 2024
Comment thread core/src/layers/mod.rs Outdated
Comment thread core/src/layers/observe/mod.rs
Comment thread core/src/layers/observe/mod.rs Outdated
Comment thread core/src/layers/prometheus.rs
Comment thread core/src/layers/prometheus.rs Outdated
/// Ok(())
/// # }
/// ```
pub fn register(self, registry: &Registry) -> Self {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need this API. PrometheusLayer::new() should be enough.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, PrometheusLayer::new should just be a helper function that uses the default config and registers the metrics with the default config to the registry.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally, I don't believe we should perform register during new unless the upstream API forbid it. Creating the layer itself should have no side effects. Everything should occur during the layer.

So PrometheusLayer::new() should just take a registry and return Self. We will perform the registry during Layer::layer.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So how about PrometheusClientLayer in #5073

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, how about this: Let's use your previous proposed builder's API.

let layer: impl Layer = PrometheusClientLayer::builder()
    .operation_bytes_buckets(vec![1.0, 2.0])
    .register(registry);

PrometheusClientLayer::builder() returns a PrometheusClientLayerBuilder. Users can only use this layer after calling register. We can provide default() if the upstream lib has default registry concept.

What do you think?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So do we need to use similar APIs in PrometheusLayer?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So do we need to use similar APIs in PrometheusLayer?

I believe this design can adapt to all metrics libraries. What do you think?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, if you agree with the design, I'll apply it to PrometheusLayer and PrometheusClientLayer.
But I'm not sure if this design will fit well with other metric layers, I'm not very optimistic after using the metrics library.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like we can call with_recorder direcly: https://docs.rs/metrics/latest/src/metrics/recorder/mod.rs.html#126

Comment thread core/src/layers/prometheus.rs Outdated
Comment thread core/src/layers/prometheus.rs Outdated
Comment thread core/src/layers/prometheus.rs Outdated
Comment thread core/src/layers/prometheus.rs Outdated
@koushiro koushiro marked this pull request as draft September 3, 2024 07:20
@koushiro koushiro changed the title refactor(layers/prometheus): provide consistent APIs refactor(layers/prometheus): provide builder APIs Sep 3, 2024
@koushiro koushiro marked this pull request as ready for review September 3, 2024 10:06
Copy link
Copy Markdown
Member

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot! This API design is really clean and easy to use. There are some nits on the public API.

Comment thread core/src/layers/prometheus.rs Outdated
Comment thread core/src/layers/prometheus.rs Outdated
Comment thread core/src/layers/prometheus.rs Outdated
Comment thread core/src/layers/prometheus.rs Outdated
let labels = OperationLabels::names(false, path_label_level);
let operation_duration_seconds = register_histogram_vec_with_registry!(
/// Register the metrics into the given registry and return a [`PrometheusLayer`].
pub fn register(self, registry: &Registry) -> PrometheusLayer {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about returning an error instead of just unwrapping?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we return prometheus::Error or opendal::Error?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

opendal only returns opendal::Error, we can wrap it inside.

@koushiro
Copy link
Copy Markdown
Member Author

koushiro commented Sep 3, 2024

There are some weird errors in CI

error[E0282]: type annotations needed
   --> src/services/icloud/core.rs:530:25
    |
530 |             Some(it) => Ok(Some(it.drivewsid.clone())),
    |                         ^^ cannot infer type of the type parameter `E` declared on the enum `Result`
    |
help: consider specifying the generic arguments
    |
530 |             Some(it) => Ok::<std::option::Option<std::string::String>, E>(Some(it.drivewsid.clone())),
    |                           +++++++++++++++++++++++++++++++++++++++++++++++

error[E0283]: type annotations needed
   --> src/services/icloud/core.rs:530:25
    |
530 |             Some(it) => Ok(Some(it.drivewsid.clone())),
    |                         ^^ cannot infer type of the type parameter `E` declared on the enum `Result`
531 |             None => Ok(None),
532 |         }?;
    |          - type must be known at this point
    |
note: multiple `impl`s satisfying `types::error::Error: std::convert::From<_>` found
   --> src/types/error.rs:418:1
    |
418 | impl From<prometheus::Error> for Error {
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    = note: and another `impl` found in the `core` crate: `impl<T> std::convert::From<T> for T;`
    = note: required for `std::result::Result<std::option::Option<std::string::String>, types::error::Error>` to implement `std::ops::FromResidual<std::result::Result<std::convert::Infallible, _>>`
help: consider specifying the generic arguments
    |
530 |             Some(it) => Ok::<std::option::Option<std::string::String>, E>(Some(it.drivewsid.clone())),
    |                           +++++++++++++++++++++++++++++++++++++++++++++++

But if I change the icloud code, these errors are gone

-        let id = match node.items.iter().find(|it| it.name == name) {
-            Some(it) => Ok(Some(it.drivewsid.clone())),
-            None => Ok(None),
-        }?;
-        Ok(id)
+       Ok(node
+          .items
+          .iter()
+          .find(|it| it.name == name)
+          .map(|it| it.drivewsid.clone()))

@Xuanwo
Copy link
Copy Markdown
Member

Xuanwo commented Sep 3, 2024

But if I change the icloud code, these errors are gone

I think it's a new Clippy lint introduced in the latest Rust version. Would you like to submit a new PR to address it?

Comment thread core/src/types/error.rs Outdated
Comment thread core/src/types/error.rs Outdated
@Xuanwo
Copy link
Copy Markdown
Member

Xuanwo commented Sep 3, 2024

9b5c096 (#5072)

I apologize for not providing a clear review. I suggest adding a parse_prometheus_error(err: prometheus::Error) -> Error as a standalone function in the Prometheus layer module itself, rather than within Error.

In this way, we keep our public types clean and easy to maintain.

Copy link
Copy Markdown
Member

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much. We are very close to merging. I found a few places I missed in the previous round.

Comment thread core/src/layers/prometheus.rs Outdated
Comment thread core/src/layers/prometheus.rs
Comment thread core/src/layers/prometheus.rs
Copy link
Copy Markdown
Member

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect, thank you for your patience!

@Xuanwo Xuanwo merged commit 70a5d7e into apache:main Sep 3, 2024
@koushiro koushiro deleted the prometheus-consistent-apis branch September 3, 2024 17:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

releases-note/refactor The PR does a refactor on code or has a title that begins with "refactor"

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants