Skip to content

KAFKA-3452: follow-up -- introduce SesssionWindows#2342

Closed
mjsax wants to merge 3 commits intoapache:trunkfrom
mjsax:kafka-3452-session-window-follow-up
Closed

KAFKA-3452: follow-up -- introduce SesssionWindows#2342
mjsax wants to merge 3 commits intoapache:trunkfrom
mjsax:kafka-3452-session-window-follow-up

Conversation

@mjsax
Copy link
Copy Markdown
Member

@mjsax mjsax commented Jan 11, 2017

  • TimeWindows represent half-open time intervals while SessionWindows represent closed time intervals

@mjsax
Copy link
Copy Markdown
Member Author

mjsax commented Jan 11, 2017

@dguy @guozhangwang @enothereska

This is a follow up to Session Windows. Because TimeWindow represent half-open time interval windows (ie, [start,end)) but sessions are defined a closed time intervals (ie, [start,end]) we should not use TimeWindow to represent a session window. Thus, we introduce SessionWindow with this update.

cf #2337 that revealed this problem

Potential issues: this PR changes abstract class Window that is strictly speaking public API.

@asfbot
Copy link
Copy Markdown

asfbot commented Jan 11, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/713/
Test PASSed (JDK 8 and Scala 2.12).

@asfbot
Copy link
Copy Markdown

asfbot commented Jan 11, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.11/715/
Test PASSed (JDK 8 and Scala 2.11).

@asfbot
Copy link
Copy Markdown

asfbot commented Jan 11, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.10/713/
Test PASSed (JDK 7 and Scala 2.10).

Copy link
Copy Markdown
Contributor

@dguy dguy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of minor comments.
Also, should we also add tests for UnlimitedWindow and TimeWindow overlap ?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make these fields final? I know they weren't before, but as you are changing this

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can. I still have #2337 open that needs to get rebased after this got merged -- will fix it there if ok.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be abstract? Can't we just have the impl of it here? i.e., the SessionWindow and TimeWindow implementations are identical.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it should be abstract because Window does not know if boundaries will be inclusive or exclusive. And implementation for SessionWindow and TimeWindow are different IMHO -- or do I miss anything -- at least, both should be different because SessionWindow does have inclusive end time while TimeWindow has exclusive end time.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. np

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pretty thorough. My only comment, and this is completely subjective so feel free to ignore. I don't really like this style of having lots of assertions in a single test. I'd much rather have single tests that are focused and test just one thing. You can then just read the test names and know how it should behave rather than having to read the code.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will change it -- still working to improve my skills to write tests. And you are my "testing hero" right now :)

@mjsax
Copy link
Copy Markdown
Member Author

mjsax commented Jan 11, 2017

Updated. Test for TimeWindow and UnlimitedWindow overlap() will be covered by #2337 after this got merged.

@asfbot
Copy link
Copy Markdown

asfbot commented Jan 11, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.11/746/
Test FAILed (JDK 8 and Scala 2.11).

Copy link
Copy Markdown
Contributor

@dguy dguy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@asfbot
Copy link
Copy Markdown

asfbot commented Jan 11, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.10/744/
Test FAILed (JDK 7 and Scala 2.10).

@asfbot
Copy link
Copy Markdown

asfbot commented Jan 11, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/744/
Test FAILed (JDK 8 and Scala 2.12).

@dguy
Copy link
Copy Markdown
Contributor

dguy commented Jan 11, 2017

@mjsax - looks like there is a checkstyle failure

@mjsax
Copy link
Copy Markdown
Member Author

mjsax commented Jan 11, 2017

@dguy @guozhangwang @enothereska Updated.

@asfbot
Copy link
Copy Markdown

asfbot commented Jan 11, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.11/755/
Test PASSed (JDK 8 and Scala 2.11).

@asfbot
Copy link
Copy Markdown

asfbot commented Jan 11, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.10/753/
Test FAILed (JDK 7 and Scala 2.10).

mjsax added 3 commits January 11, 2017 13:18
 - TimeWindows represent half-open time intervals while SessionWindows represent closed time intervals
@mjsax mjsax force-pushed the kafka-3452-session-window-follow-up branch from c348e03 to 6c3d5af Compare January 11, 2017 21:19
@asfbot
Copy link
Copy Markdown

asfbot commented Jan 11, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/753/
Test PASSed (JDK 8 and Scala 2.12).

@asfbot
Copy link
Copy Markdown

asfbot commented Jan 11, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/758/
Test FAILed (JDK 8 and Scala 2.12).

@asfbot
Copy link
Copy Markdown

asfbot commented Jan 11, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.10/758/
Test PASSed (JDK 7 and Scala 2.10).

@asfbot
Copy link
Copy Markdown

asfbot commented Jan 11, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.11/760/
Test FAILed (JDK 8 and Scala 2.11).

@mjsax
Copy link
Copy Markdown
Member Author

mjsax commented Jan 11, 2017

Failing tests are unrelated.

@guozhangwang
Copy link
Copy Markdown
Contributor

retest this please

@asfbot
Copy link
Copy Markdown

asfbot commented Jan 12, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/768/
Test PASSed (JDK 8 and Scala 2.12).

@asfbot
Copy link
Copy Markdown

asfbot commented Jan 12, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.10/768/
Test PASSed (JDK 7 and Scala 2.10).

@asfbot
Copy link
Copy Markdown

asfbot commented Jan 12, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.11/770/
Test FAILed (JDK 8 and Scala 2.11).

@guozhangwang
Copy link
Copy Markdown
Contributor

LGTM and merged to trunk.

@asfgit asfgit closed this in 3c60511 Jan 12, 2017
@mjsax mjsax deleted the kafka-3452-session-window-follow-up branch January 12, 2017 17:59
soenkeliebau pushed a commit to soenkeliebau/kafka that referenced this pull request Feb 7, 2017
 - TimeWindows represent half-open time intervals while SessionWindows represent closed time intervals

Author: Matthias J. Sax <matthias@confluent.io>

Reviewers: Damian Guy <damian.guy@gmail.com>, Guozhang Wang <wangguoz@gmail.com>

Closes apache#2342 from mjsax/kafka-3452-session-window-follow-up
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants