KAFKA-14717 KafkaStreams can' get running if the rebalance happens be…#13248
KAFKA-14717 KafkaStreams can' get running if the rebalance happens be…#13248guozhangwang merged 4 commits intoapache:trunkfrom
Conversation
…fore StreamThread gets shutdown completely
guozhangwang
left a comment
guozhangwang
left a comment
There was a problem hiding this comment.
Thanks @chia7712 for the find! After digging it I agree this is a real issue. Though I have some questions about the fix itself, and also about the test covering this scenario. Please let me know what do you think?
| final Properties prop = new Properties(); | ||
| prop.putAll(properties); | ||
| // make rebalance happen quickly | ||
| prop.put(ConsumerConfig.HEARTBEAT_INTERVAL_MS_CONFIG, 200); |
There was a problem hiding this comment.
Is there a better way to test this scenario than relying on the real time (and hence is time dependendent)? More specifically, I'm looking for a test case which would 100% fail without the fix, and would 100% pass with the fix. While this test seems would be pass some times even without the fix, is that right?
There was a problem hiding this comment.
While this test seems would be pass some times even without the fix, is that right?
you are right. I have updated the test to make sure it is always failed without the fix.
|
LGTM. Merged to trunk. |
2 participants
I noticed this issue when tracing #12590
StreamThread closes the consumer before changing state to DEAD. If the partition rebalance happens quickly, the other StreamThreads can't change KafkaStream state from REBALANCING to RUNNING since there is a PENDING_SHUTDOWN StreamThread
Committer Checklist (excluded from commit message)