KAFKA-9844; Maximum number of members within a group is not always enforced due to a race condition in join group#8454
Conversation
…forced due to a race condition in join group
|
I think that this is the root cause of https://issues.apache.org/jira/browse/KAFKA-7965. |
|
ok to test |
abbccdda
left a comment
abbccdda
left a comment
There was a problem hiding this comment.
Thanks for the fix! Although I don't think this alone helps close the gap for the flaky test itself as I have also seen cases where we reject more members than necessary. Could we get a unit test for the patch?
|
Thanks @dajac. This fix LGTM. It might be worth taking a look at |
|
@abbccdda @hachikuji I have added a test in |
|
ok to test |
|
retest this please |
1 similar comment
|
retest this please |
hachikuji
left a comment
hachikuji
left a comment
There was a problem hiding this comment.
Thanks for the test case. Left just one minor comment.
| */ | ||
| def getGroup(groupId: String): Option[GroupMetadata] = { | ||
| Option(groupMetadataCache.get(groupId)) | ||
| def getGroup(groupId: String, createIfNotExist: Boolean = false): Option[GroupMetadata] = { |
There was a problem hiding this comment.
Optional arguments are best avoided. Maybe we could split this into two methods getOrCreateGroup and getGroup?
There was a problem hiding this comment.
Sure. I went with the following method getOrMaybeCreateGroup which still takes the createIfNotExist argument in order to keep the calling side concise. It avoids having to switch between the two on the calling side which makes the error handling a little easier to follow. This is perhaps what you had in mind already.
|
retest this please |
|
ok to test |
|
retest this please |
|
@hachikuji Could we get this one merged? |
|
retest this please |
…t-for-generated-requests * apache-github/trunk: (366 commits) MINOR: Improve producer test BufferPoolTest#testCloseNotifyWaiters. (apache#7982) MINOR: document how to escape json parameters to ducktape tests (apache#8546) KAFKA-9885; Evict last members of a group when the maximum allowed is reached (apache#8525) KAFKA-9866: Avoid election for topics where preferred leader is not in ISR (apache#8524) KAFKA-9839; Broker should accept control requests with newer broker epoch (apache#8509) KAKFA-9612: Add an option to kafka-configs.sh to add configs from a prop file (KIP-574) MINOR: Partition is under reassignment when adding and removing (apache#8364) MINOR: reduce allocations in log start and recovery checkpoints (apache#8467) MINOR: Remove unused foreign-key join class (apache#8547) HOTFIX: Fix broker bounce system tests (apache#8532) KAFKA-9704: Fix the issue z/OS won't let us resize file when mmap. (apache#8224) KAFKA-8639: Replace AddPartitionsToTxn with Automated Protocol (apache#8326) MINOR: equals() should compare all fields for generated classes (apache#8539) KAFKA-9844; Fix race condition which allows more than maximum number of members(apache#8454) KAFKA-9823: Remember the sent generation for the coordinator request (apache#8445) KAFKA-9883: Add better error message when REST API forwards a request and leader is not known (apache#8536) KAFKA-9907: Switch default build to Scala 2.13 (apache#8537) MINOR: Some html fixes in Streams DSL documentation (apache#8503) MINOR: Enable fatal warnings with scala 2.13 (apache#8429) KAFKA-9852: Change the max duration that calls to the buffer pool can block from 2000ms to 10ms to reduce overall test runtime (apache#8464) ...
3 participants
This patch fixes a race condition in the join group request handling which sometimes results in not enforcing the maximum number of members allowed in a group. The JIRA provides an example: https://issues.apache.org/jira/browse/KAFKA-9844
Committer Checklist (excluded from commit message)