KAFKA-9483; Add Scala KStream#toTable to the Streams DSL

[highluck]

Add Scala KStream#toTable to the Streams DSL
https://issues.apache.org/jira/browse/KAFKA-9483

Committer Checklist (excluded from commit message)

• Verify design and implementation
• Verify test coverage and CI build status
• Verify documentation (including upgrade notes)

[mjsax]

LGTM.

@vvcephei @bbejeck -- I am not fluent in Scala -- can you have a second look?

[bbejeck]

LGMT pending passing build

[bbejeck]

Retest this please.

[mjsax]

Java 11: KStreamImplTest.shouldSupportTriggerMaterializedWithKTableFromKStream (should be covered via #8027 -- did not create a ticket)

Java 8:

kafka.admin.DeleteConsumerGroupsTest.testDeleteCmdWithMixOfSuccessAndError
kafka.admin.ResetConsumerGroupOffsetTest.testResetOffsetsExportImportPlan

We can trigger retesting after John's review.

[mjsax]

I just realized, that Bill review and approved already...

Retest this please.

[mjsax]

Both tests failed. Result are not available any longer.

Retest this please.

[mjsax]

I need to cycle back to this quicker. Same as before. Both tests failed but result are not available any longer.

Retest this please.

[mjsax]

Java 8:

kafka.admin.DeleteConsumerGroupsTest.testDeleteCmdWithMixOfSuccessAndError

Java 11:

kafka.admin.DescribeConsumerGroupTest.testDescribeGroupWithShortInitializationTimeout

Java 8 test failure is the same as above. Retest this please.

[mjsax]

@bbejeck @vvcephei @guozhangwang The Scala part of the API change was not explicitly discussed in the KIP -- do you think this is an issue and we should try to include such a discussion for all API changes?

Also, this PR is technically an API change, hence, I am wondering if it would be ok to cherry-pick to 2.5 branch or not? The feature itself was merged on time -- I personally think it would be ok to cherry-pick, but would like to double check what you think?

[mjsax]

Java 8 timed out.
Java 11:

kafka.admin.ResetConsumerGroupOffsetTest.testResetOffsetsExportImportPlan

Retest this please.

[mjsax]

Java 8:

kafka.admin.DescribeConsumerGroupTest.testDescribeGroupMembersWithShortInitializationTimeout

Java 11:

 kafka.admin.DeleteConsumerGroupsTest > testDeleteCmdWithMixOfSuccessAndError

I guess we can merge this?

[highluck]

@mjsax
Thank you for your care!
But, No problem in my local environment

[vvcephei]

Hey all, really sorry about this, but I just now noticed a significant issue in the test.

Regarding @mjsax 's earlier question, I can't believe we missed the scala API again in the KIP... We should just update the KIP and send a note to the vote thread. Also, we should cherry-pick to 2.5.

[bbejeck]

Regarding @mjsax 's earlier question, I can't believe we missed the scala API again in the KIP... We should just update the KIP and send a note to the vote thread. Also, we should cherry-pick to 2.5.

+1

[highluck]

@vvcephei
thanks for review !
Updated the KIP and updated the test.

retest this please

[vvcephei]

Thanks @highluck

[vvcephei]

Ok to test.

[vvcephei]

Retest this, please.

[mjsax]

Both runs passed. Merging.

[mjsax]

Merged to trunk and cherry-picked to 2.5.

Thanks for the PR @highluck!