Feature request: Reuse namespace in Metrics.flushSingleMetric and other limitations

[humanzz]

Use case

As a user of Powertools v1, and considering moving to v2, I found the withMetricsLogger now replaced with flushSingleMetric. I see 2 issues

1. The current overloads, requires setting namespace, even though previously I've been relying on the namespace being whatever was already set (e.g. via powertools environment variables). I find this a degradation in UX if - which I think is a typical use case - I wanna use the same namespace
2. I also had use cases for withMetricsLogger where I add multiple metrics, which share an overriden dimensionset. Now this needs to be replaced with multiple calls to flushSingleMetric

I think the refactor/abstractions led to some degradations when it comes to withMetricsLogger. Moreover, LambdaMetricsAspect do set a few other properties/metadata that no longer seem to be available in instances emitted by flushSingleMetric such as trace and function id.

Solution/User Experience

For 1, one solution would be to add flushSingleMetric overloads that don't take a namespace, and to update EmfMetricsLogger to treat the namespace parameter as nullable and a potential override to its this.namespace

For 2, maybe Metrics should get a new method flushMetrics with takes a Consumer<Metrics>, then one is more free about using default dimensions/namespace, or overriding them and adding multiple metrics.

Alternative solutions

N/A

Acknowledgment

• This feature request meets Powertools for AWS Lambda (Java) Tenets
• Should this be considered in other Powertools for AWS Lambda languages? i.e. Python, TypeScript, and .NET

Future readers

Please react with 👍 and your use case to help us understand customer demand.

[leandrodamascena]

Hey @humanzz thanks for opening this issue and explaining more about the Developer Experience you expect from Powertools.

Use case

As a user of Powertools v1, and considering moving to v2, I found the withMetricsLogger now replaced with flushSingleMetric. I see 2 issues

1. The current overloads, requires setting namespace, even though previously I've been relying on the namespace being whatever was already set (e.g. via powertools environment variables). I find this a degradation in UX if - which I think is a typical use case - I wanna use the same namespace

Actually I think Java V2 aligns with the API for SingleMetrics in other runtimes, like Python and TypeScript. The mental model for a single metric is to have everything specific to that metric, like dimensions, values, and namespace. I'm open to discussing this and seeing if we can adapt this behavior or if we need to improve our documentation. Can you explain a bit more about using SingleMetrics with the same namespace? Shouldn't this be a use case for regular metric instances?

3. I also had use cases for withMetricsLogger where I add multiple metrics, which share an overriden dimensionset. Now this needs to be replaced with multiple calls to flushSingleMetric>
   I think the refactor/abstractions led to some degradations when it comes to withMetricsLogger. Moreover, LambdaMetricsAspect do set a few other properties/metadata that no longer seem to be available in instances emitted by flushSingleMetric such as trace and function id.

Can you give us your opinion here, @phipag?

Solution/User Experience

For 1, one solution would be to add flushSingleMetric overloads that don't take a namespace, and to update EmfMetricsLogger to treat the namespace parameter as nullable and a potential override to its this.namespace

For 2, maybe Metrics should get a new method flushMetrics with takes a Consumer<Metrics>, then one is more free about using default dimensions/namespace, or overriding them and adding multiple metrics.

Thanks for suggesting a better experience, but I think we have something to discuss before we decide which way to go.

[phipag]

Can you explain a bit more about using SingleMetrics with the same namespace? Shouldn't this be a use case for regular metric instances?

I would love to learn about such a use-case and how you solve it in v1 at the moment.

I also had use cases for withMetricsLogger where I add multiple metrics, which share an overriden dimensionset. Now this needs to be replaced with multiple calls to flushSingleMetric

This is a good call and indeed not possible anymore in v2. I see Python has

with single_metric(name="MySingleMetric", unit=MetricUnit.Count, value=1) as metric:
        metric.add_dimension(name="environment", value=STAGE)

When I implementing flushSingleMetric I was following the fact that emission of such single metrics should be single indeed – but I recognize there is a use-case for flushing multiple metrics with a different configuration in one go instead of calling flush every single time (e.g. in a for-loop). Inline with Java v2 naming conventions using the flush* syntax, I can image to introduce a new method such as this to support the use-case of flushing multiple custom metrics with a single call to flush:

    /**
     * Execute custom metrics collection with a fresh metrics instance.
     * The provided function receives a new Metrics instance that can be configured
     * independently (namespace, dimensions, etc.) and used to collect multiple metrics.
     * All metrics are automatically flushed after the function execution.
     *
     * @param metricsFunction function that receives a fresh Metrics instance for custom metric collection
     */
    void flushCustomMetrics(java.util.function.Consumer<Metrics> metricsFunction);

The user-experience will look similar to this where metrics is the regular Metrics Singleton returned from the MetricsFactory:

metrics.flushCustomMetrics(customMetrics -> {
    customMetrics.setNamespace("MyCustomNamespace");
    customMetrics.addDimension("Environment", "Production");
    customMetrics.addMetric("CustomMetric1", 42.0, MetricUnit.COUNT);
    customMetrics.addMetric("CustomMetric2", 3.14, MetricUnit.SECONDS);
    // No need to call flush() - it's done automatically
});

set a few other properties/metadata that no longer seem to be available in instances emitted by flushSingleMetric such as trace and function id

Was this available in Java v1 (@humanzz?). How do you solve this in Python (@leandrodamascena?).

[leandrodamascena]

In Python, we don't define anything related to Tracerid, FunctionID, or any other Lambda function metadata when using metrics. In Metrics, we only add namespace, name, value, and dimensions that customer explicitly set it. I think PT Java v1 at some point had metrics published with function metadata, but I could be wrong.

[phipag]

Yes, I believe it was published with function metadata in v1 due to exposing the interface of the Java EMF library directly. However, we still set the trace id for example in LambdaMetricsAspect for all regular metrics and the function name for the cold start metric.