KAFKA-10082: Fix the failed testMultiConsumerStickyAssignment#8777
Merged
guozhangwang merged 2 commits intoapache:trunkfrom Jun 2, 2020
Merged
KAFKA-10082: Fix the failed testMultiConsumerStickyAssignment#8777guozhangwang merged 2 commits intoapache:trunkfrom
guozhangwang merged 2 commits intoapache:trunkfrom
Conversation
…fy the logic error in allSubscriptionsEqual method
Member
Author
|
@ableegoldman , could you check this PR to fix the failed tests? Thanks. |
3 tasks
ableegoldman
approved these changes
Jun 2, 2020
Member
ableegoldman
left a comment
ableegoldman
left a comment
There was a problem hiding this comment.
Oh man, this is a bad one. I refactored this a bit right before it was merged and guess I never ran the full suite of tests a final time 😅
Thanks for all the fixes!
Member
|
cc @mjsax @guozhangwang , should be cherrypicked to 2.6, 2.5, and 2.4 |
guozhangwang
pushed a commit
that referenced
this pull request
Jun 2, 2020
Fix the failed testMultiConsumerStickyAssignment by modifying the logic error in allSubscriptionsEqual method. We will create the consumerToOwnedPartitions to keep the set of previously owned partitions encoded in the Subscription. It's our basis to do the reassignment. In the allSubscriptionsEqual, we'll get the member generation of the subscription, and remove all previously owned partitions as invalid if the current generation is higher. However, the logic before my fix, will remove the current highest member out of the consumerToOwnedPartitions, which should be kept because it's the current higher generation member. Fix this logic error. Reviewers: A. Sophie Blee-Goldman <sophie@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
guozhangwang
pushed a commit
that referenced
this pull request
Jun 2, 2020
Fix the failed testMultiConsumerStickyAssignment by modifying the logic error in allSubscriptionsEqual method. We will create the consumerToOwnedPartitions to keep the set of previously owned partitions encoded in the Subscription. It's our basis to do the reassignment. In the allSubscriptionsEqual, we'll get the member generation of the subscription, and remove all previously owned partitions as invalid if the current generation is higher. However, the logic before my fix, will remove the current highest member out of the consumerToOwnedPartitions, which should be kept because it's the current higher generation member. Fix this logic error. Reviewers: A. Sophie Blee-Goldman <sophie@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
guozhangwang
pushed a commit
that referenced
this pull request
Jun 2, 2020
Fix the failed testMultiConsumerStickyAssignment by modifying the logic error in allSubscriptionsEqual method. We will create the consumerToOwnedPartitions to keep the set of previously owned partitions encoded in the Subscription. It's our basis to do the reassignment. In the allSubscriptionsEqual, we'll get the member generation of the subscription, and remove all previously owned partitions as invalid if the current generation is higher. However, the logic before my fix, will remove the current highest member out of the consumerToOwnedPartitions, which should be kept because it's the current higher generation member. Fix this logic error. Reviewers: A. Sophie Blee-Goldman <sophie@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
Contributor
|
Cherry-picked to 2.6 / 2.5 / 2.4 |
ijuma
added a commit
to confluentinc/kafka
that referenced
this pull request
Jun 3, 2020
* apache-github/2.6: (32 commits) KAFKA-10083: fix failed testReassignmentWithRandomSubscriptionsAndChanges tests (apache#8786) KAFKA-9945: TopicCommand should support --if-exists and --if-not-exists when --bootstrap-server is used (apache#8737) KAFKA-9320: Enable TLSv1.3 by default (KIP-573) (apache#8695) KAFKA-10082: Fix the failed testMultiConsumerStickyAssignment (apache#8777) MINOR: Remove unused variable to fix spotBugs failure (apache#8779) MINOR: ChangelogReader should poll for duration 0 for standby restore (apache#8773) KAFKA-10030: Allow fetching a key from a single partition (apache#8706) Kafka-10064 Add documentation for KIP-571 (apache#8760) MINOR: Code cleanup and assertion message fixes in Connect integration tests (apache#8750) KAFKA-9987: optimize sticky assignment algorithm for same-subscription case (apache#8668) KAFKA-9392; Clarify deleteAcls javadoc and add test for create/delete timing (apache#7956) KAFKA-10074: Improve performance of `matchingAcls` (apache#8769) KAFKA-9494; Include additional metadata information in DescribeConfig response (KIP-569) (apache#8723) KAFKA-10056; Ensure consumer metadata contains new topics on subscription change (apache#8739) KAFKA-10029; Don't update completedReceives when channels are closed to avoid ConcurrentModificationException (apache#8705) KAFKA-10061; Fix flaky `ReassignPartitionsIntegrationTest.testCancellation` (apache#8749) KAFKA-9130; KIP-518 Allow listing consumer groups per state (apache#8238) KAFKA-9501: convert between active and standby without closing stores (apache#8248) MINOR: Relax Percentiles test (apache#8748) MINOR: regression test for task assignor config (apache#8743) ...
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix the failed
testMultiConsumerStickyAssignmentby modifying the logic error inallSubscriptionsEqualmethod.We will create the
consumerToOwnedPartitionsto keep the set of previously owned partitions encoded in the Subscription. It's our basis to do the reassignment. In theallSubscriptionsEqual, we'll get the member generation of the subscription, and remove all previously owned partitions as invalid if the current generation is higher. However, the logic before my fix, will remove the current highest member out of theconsumerToOwnedPartitions, which should be kept because it's the current higher generation member. Fix this logic error. BTW, it's a nice improvement in KAFKA-9987!More detailed description of your change,
if necessary. The PR title and PR message become
the squashed commit message, so use a separate
comment to ping reviewers.
Summary of testing strategy (including rationale)
for the feature or bug fix. Unit and/or integration
tests are expected for any behaviour change and
system tests should be considered for larger changes.
Committer Checklist (excluded from commit message)