[Bug]: Samza AsyncDoFnRunnerTest.testSimplePipeline is flaky (now disabled)

[kennknowles]

What happened?

On an unrelated PR, this failure occurred: https://ci-beam.apache.org/job/beam_PreCommit_Java_Commit/24482/testReport/junit/org.apache.beam.runners.samza.runtime/AsyncDoFnRunnerTest/testSimplePipeline/

java.lang.AssertionError: MapElements/Map/ParMultiDo(Anonymous).output: 
Expected: iterable with items [<1>, <4>, <9>, <16>, <25>] in any order
     but: no item matches: <4>, <9>, <16>, <25> in [<1>]
	at org.apache.beam.sdk.testing.PAssert$PAssertionSite.capture(PAssert.java:175)
	at org.apache.beam.sdk.testing.PAssert.that(PAssert.java:460)
	at org.apache.beam.sdk.testing.PAssert.that(PAssert.java:452)
	at org.apache.beam.runners.samza.runtime.AsyncDoFnRunnerTest.testSimplePipeline(AsyncDoFnRunnerTest.java:82)

Issue Priority

Priority: 1

Issue Component

Component: runner-samza

[kennknowles]

https://ci-beam.apache.org/job/beam_PreCommit_Java_Phrase/5806/ could be another

[kennknowles]

@xinyuiscool do you have any insight into this flaky test?

[xinyuiscool]

@kennknowles : yes, it was a bug in Samza that the shutdown doesn't wait for all the futures being completed (it's not expecting bounded streams with async processing). I will disable these tests for now and reenable after samza runner has the fix.

[xinyuiscool]

Created the pr for turn off the tests: #24481