KAFKA-6399: Remove Streams max.poll.interval override#6509
Merged
bbejeck merged 3 commits intoapache:trunkfrom Apr 4, 2019
vvcephei:KAFKA-6399-reduce-streams-poll-ms
Merged
KAFKA-6399: Remove Streams max.poll.interval override#6509bbejeck merged 3 commits intoapache:trunkfrom vvcephei:KAFKA-6399-reduce-streams-poll-ms
bbejeck merged 3 commits intoapache:trunkfrom
vvcephei:KAFKA-6399-reduce-streams-poll-ms
Conversation
Contributor
Author
|
Can you take a quick look at this, please? Thanks, |
| tempConsumerDefaultOverrides.put(ConsumerConfig.MAX_POLL_INTERVAL_MS_CONFIG, Integer.toString(Integer.MAX_VALUE)); | ||
| tempConsumerDefaultOverrides.put( | ||
| ConsumerConfig.MAX_POLL_INTERVAL_MS_CONFIG, | ||
| Long.toString(Duration.ofMinutes(1).toMillis()) |
Contributor
There was a problem hiding this comment.
ConsumerConfigs.MAX_POLL_INTERVAL_MS_CONFIG default value is 5 minutes, so I think it is not necessary to further decrease it to 1 minute in Streams; maybe we can just remove this override to defer to Consumer's default value?
BTW we should have a tiny KIP for this default value change.
Contributor
Author
There was a problem hiding this comment.
Ok, I've taken your recommendation and also created KIP-442: https://cwiki.apache.org/confluence/display/KAFKA/KIP-442%3A+Return+to+default+max+poll+interval+in+Streams (https://cwiki.apache.org/confluence/x/1COGBg)
vvcephei
changed the title
Contributor
Contributor
Author
|
Thanks, all. I've closed voting and marked the KIP as accepted. |
Member
|
Merged #6509 into trunk. |
jarekr
pushed a commit
to confluentinc/kafka
that referenced
this pull request
Apr 18, 2019
* apache/trunk: MINOR: Add security considerations for remote JMX in Kafka docs (apache#6544) MINOR: Remove redundant access specifiers from metrics interfaces (apache#6527) MINOR: Correct KStream documentation (apache#6552) KAFKA-8013; Avoid underflow when reading a Struct from a partially correct buffer (apache#6340) KAFKA-8058: Fix ConnectClusterStateImpl.connectors() method (apache#6384) MINOR: Move common consumer tests out of abstract consumer class (apache#6548) KAFKA-8168; Add a generated ApiMessageType class KAFKA-7893; Refactor ConsumerBounceTest to reuse functionality from BaseConsumerTest (apache#6238) MINOR: Tighten up metadata upgrade test (apache#6531) KAFKA-8190; Don't update keystore modification time during validation (apache#6539) MINOR: Fixed a few warning in core and connects (apache#6545) KAFKA-7904; Add AtMinIsr partition metric and TopicCommand option (KIP-427) MINOR: fix throttling and status in ConnectionStressWorker KAFKA-8090: Use automatic RPC generation in ControlledShutdown KAFKA-6399: Remove Streams max.poll.interval override (apache#6509) KAFKA-8126: Flaky Test org.apache.kafka.connect.runtime.WorkerTest.testAddRemoveTask (apache#6475) HOTFIX: Update unit test for KIP-443 KAFKA-7190: KIP-443; Remove streams overrides on repartition topics (apache#6511) KAFKA-8183: Add retries to WorkerUtils#verifyTopics (apache#6532) KAFKA-8181: Removed Avro topic from TOC on kafka (apache#6529)
pengxiaolong
pushed a commit
to pengxiaolong/kafka
that referenced
this pull request
Jun 14, 2019
Since we now call poll during restore, we can decrease the timeout to a reasonable value, which should help Streams make progress if threads get stuck. Reviewers: Guozhang Wang <wangguoz@gmail.com>, Bill Bejeck <bbejeck@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Since we now call
pollduring restore, we can decrease the timeoutto a reasonable value, which should help Streams make progress if
threads get stuck.
Committer Checklist (excluded from commit message)