Skip to content

[Bug][prism]: Support receiving Java's CoGBKList Result #28544

@lostluck

Description

@lostluck

What happened?

Run the TestXlang_CoGroupBy test from the Go validates runner suite.

Data gets corrupted, and if the Go SDK does loopback execution the passert returns the following:

        actual PCollection does not match expected values
        =========
        0 correct entries (present in both)
        =========
        2 unexpected entries (present in actual, missing in expected)
        +++
        1964988:[]
        +++
        1:[]
        =========
        2 missing entries (missing in actual, present in expected)
        ---
        0:[1 2 4]
        ---
        1:[3 5 6]

As this is only happening in prism (other runners seem to cope), either prism's unfused execution is causing corruption on the Java side, OR it's prism receiving it is causing the corruption. Either way, it needs to be fixed so the pipeline passes.

The test in question is here:

func TestXLang_CoGroupBy(t *testing.T) {

It needs to be unfilered from the prism run here though:

"TestXLang_CoGroupBy",

The following action workflow runs the suite:

https://github.com/apache/beam/actions/workflows/beam_PreCommit_GoPrism.yml

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Samza Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions