KAFKA-7895: fix stream-time reckoning for Suppress (2.2)#6286
KAFKA-7895: fix stream-time reckoning for Suppress (2.2)#6286bbejeck merged 6 commits intoapache:2.2from vvcephei:KAFKA-7895-fix-suppress-2.2
Conversation
|
Kicked off system tests: https://jenkins.confluent.io/job/system-test-kafka-branch-builder/2355 |
|
@vvcephei Jenkins failed with checkstyle errors. |
|
Goodness... I had to wrestle with the driver a bit, since I didn't backport the test for the driver, and since I'm not able to run the system tests locally. I got the driver system test to pass: Waiting on the full suite of tests: |
mjsax
left a comment
There was a problem hiding this comment.
LGTM.
Waiting for system tests to finish before merging.
bbejeck
left a comment
There was a problem hiding this comment.
LGTM, system test failures (broker upgrade-downgrade) are unrelated and are a known issue
Even within a Task, different Processors have different perceptions of time, due to record caching on stores and in suppression itself, and in general, due to any processor logic that may hold onto records arbitrarily and emit them later. Thanks to this, we can't rely on the whole task existing in the same "instant" of stream-time. The solution is for each processor node that cares about stream-time to track it independently. Reviewers: Guozhang Wang <wangguoz@gmail.com>, Bill Bejeck <bbejeck@gmail.com>
Even within a Task, different Processors have different perceptions
of time, due to record caching on stores and in suppression itself,
and in general, due to any processor logic that may hold onto
records arbitrarily and emit them later. Thanks to this, we can't rely
on the whole task existing in the same "instant" of stream-time. The
solution is for each processor node that cares about stream-time to
track it independently.
See also #6278
Committer Checklist (excluded from commit message)