Skip to content

[prism] Support Attempted Metrics #31926

@lostluck

Description

@lostluck

Prism doesn't currently populate Attempted metrics, just committed metrics, when returning metrics through Job Logs.

In principle, for completed jobs, the values should be identical. They should only be different while a Job is executing.

The short version: Committed metrics should be all metrics from ProcessBundleResponses, while tentative metrics are those plus anything from a ProcessBundleProgressResponse.

Metrics are accumulated here.
https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/runners/prism/internal/jobservices/metrics.go#L483

This would allow the following Java Validates Runner tests to pass:

MetricsTest$AttemptedMetricTests.testAttemptedDistributionMetrics
MetricsTest$AttemptedMetricTests.testAttemptedCounterMetrics

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions