Skip to content

MINOR: Add test demonstrating re-use of KGroupedStream with Optimizations enabled#6050

Merged
guozhangwang merged 1 commit intoapache:trunkfrom
bbejeck:MINOR_add_test_for_reusing_KGroupedStream_with_optimizations_turned_on
Dec 22, 2018
Merged

MINOR: Add test demonstrating re-use of KGroupedStream with Optimizations enabled#6050
guozhangwang merged 1 commit intoapache:trunkfrom
bbejeck:MINOR_add_test_for_reusing_KGroupedStream_with_optimizations_turned_on

Conversation

@bbejeck
Copy link
Copy Markdown
Member

@bbejeck bbejeck commented Dec 19, 2018

Right now if a repartition is required and users choose to name the repartition topic for an aggregation
i.e kGroupedStream = builder.<String, String>stream("topic").selectKey((k, v) -> k).groupByKey(Grouped.as("grouping")); The resulting KGroupedStream can't be reused
with optimizations are disabled, as Streams will attempt to create two repartiton topics with the same name.

However, if optimizations are enabled then the resulting KGroupedStream can be re-used
For example the following will work if optimizations are enabled.

kGroupedStream = builder.<String, String>stream("topic")
                             .selectKey((k, v) -> k)
                             .groupByKey(Grouped.as("grouping"));

kGroupedStream.windowedBy(TimeWindows.of(Duration.ofMillis(10L))).count();
kGroupedStream.windowedBy(TimeWindows.of(Duration.ofMillis(30L))).count();

This PR provides a unit test proving as much.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@bbejeck
Copy link
Copy Markdown
Member Author

bbejeck commented Dec 19, 2018

ping @guozhangwang, @mjsax, and @vvcephei for reviews

Copy link
Copy Markdown
Member

@mjsax mjsax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@guozhangwang guozhangwang merged commit 4616c0a into apache:trunk Dec 22, 2018
pengxiaolong pushed a commit to pengxiaolong/kafka that referenced this pull request Jun 14, 2019
…ions enabled (apache#6050)

Right now if a repartition is required and users choose to name the repartition topic for an aggregation i.e. kGroupedStream = builder.<String, String>stream("topic").selectKey((k, v) -> k).groupByKey(Grouped.as("grouping")); The resulting KGroupedStream can't be reused
with optimizations are disabled, as Streams will attempt to create two repartiton topics with the same name.

However, if optimizations are enabled then the resulting KGroupedStream can be re-used
For example the following will work if optimizations are enabled.

This PR provides a unit test proving as much.

Reviewers: Matthias J. Sax <mjsax@apache.org>, Guozhang Wang <wangguoz@gmail.com>
@bbejeck bbejeck deleted the MINOR_add_test_for_reusing_KGroupedStream_with_optimizations_turned_on branch July 10, 2024 12:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants