-
Notifications
You must be signed in to change notification settings - Fork 4.5k
Description
Prism doesn't currently populate Attempted metrics, just committed metrics, when returning metrics through Job Logs.
In principle, for completed jobs, the values should be identical. They should only be different while a Job is executing.
The short version: Committed metrics should be all metrics from ProcessBundleResponses, while tentative metrics are those plus anything from a ProcessBundleProgressResponse.
Metrics are accumulated here.
https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/runners/prism/internal/jobservices/metrics.go#L483
This would allow the following Java Validates Runner tests to pass:
MetricsTest$AttemptedMetricTests.testAttemptedDistributionMetrics
MetricsTest$AttemptedMetricTests.testAttemptedCounterMetrics