[Feature Request]: Support returning back BoundedTrie in metric result

[evelyn97]

What would you like to happen?

Dataflow service side is working on returning the BoundedTrie metrics.

Run into the following error:

java.lang.ClassCastException: com.google.api.client.util.ArrayMap cannot be cast to org.apache.beam.model.pipeline.v1.MetricsApi$BoundedTrie
    at org.apache.beam.runners.dataflow.DataflowMetrics$DataflowMetricResultExtractor.getBoundedTrieValue (DataflowMetrics.java:212)
    at org.apache.beam.runners.dataflow.DataflowMetrics$DataflowMetricResultExtractor.addMetricResult (DataflowMetrics.java:179)
    at org.apache.beam.runners.dataflow.DataflowMetrics$DataflowMetricQueryResultsFactory.build (DataflowMetrics.java:406)
    at org.apache.beam.runners.dataflow.DataflowMetrics.populateMetricQueryResults (DataflowMetrics.java:100)
    at org.apache.beam.runners.dataflow.DataflowMetrics.queryMetrics (DataflowMetrics.java:120)
    at org.apache.beam.sdk.metrics.Lineage.getLineageQueryResults (Lineage.java:249)
    at org.apache.beam.sdk.metrics.Lineage.query (Lineage.java:161)
    at org.apache.beam.sdk.metrics.Lineage.query (Lineage.java:197)
    at org.apache.beam.examples.GCSStreamingPipeline.main (GCSStreamingPipeline.java:90)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282)
    at java.lang.Thread.run (Thread.java:750)

Changes needed here at https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowMetrics.java#L212 to translate from the struct returned by Dataflow into the BoundedTrie proto.

Issue Priority

Priority: 2 (default / most feature requests should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Infrastructure
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[TanuSharma2511]

I would like to work on this request. @evelyn97 Is it possible for you to share repro of this issue ?

[liferoad]

@evelyn97 is working on this now.

[github-actions]

This issue has been marked as stale due to 150 days of inactivity. It will be closed in 30 days if no further activity occurs. If you think that’s incorrect or this issue still needs to be addressed, please simply write any comment. If closed, you can reopen the issue at any time. Thank you for your contributions.

[Abacn]

fixed by #34738