KAFKA-8011: Fix for race condition causing ConcurrentModificationException#6338
Merged
bbejeck merged 1 commit intoapache:trunkfrom Feb 28, 2019
Merged
Conversation
Member
Author
|
\cc @vvcephei @ableegoldman for reviews |
guozhangwang
approved these changes
Feb 28, 2019
Contributor
guozhangwang
left a comment
guozhangwang
left a comment
There was a problem hiding this comment.
Thanks @bbejeck ! LGTM.
Contributor
|
@bbejeck please feel free to cherry-pick to older branches as well. |
Member
Author
|
Merged #6338 to trunk |
bbejeck
added a commit
that referenced
this pull request
Feb 28, 2019
…ception (#6338) In the RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated() and RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenDeleted() a race condition exists where the ConsumerRebalanceListener in the test modifies the list of subscribed topics when the condition for the test success is comparing the same array instance against expected values. This PR should fix this race condition by using a CopyOnWriteArrayList which guarantees safe traversal of the list even when a concurrent modification is happening. Reviewers: Guozhang Wang <wangguoz@gmail.com>
bbejeck
added a commit
that referenced
this pull request
Feb 28, 2019
…ception (#6338) In the RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated() and RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenDeleted() a race condition exists where the ConsumerRebalanceListener in the test modifies the list of subscribed topics when the condition for the test success is comparing the same array instance against expected values. This PR should fix this race condition by using a CopyOnWriteArrayList which guarantees safe traversal of the list even when a concurrent modification is happening. Reviewers: Guozhang Wang <wangguoz@gmail.com>
bbejeck
added a commit
that referenced
this pull request
Mar 1, 2019
…ception (#6338) In the RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated() and RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenDeleted() a race condition exists where the ConsumerRebalanceListener in the test modifies the list of subscribed topics when the condition for the test success is comparing the same array instance against expected values. This PR should fix this race condition by using a CopyOnWriteArrayList which guarantees safe traversal of the list even when a concurrent modification is happening. Reviewers: Guozhang Wang <wangguoz@gmail.com>
bbejeck
added a commit
that referenced
this pull request
Mar 1, 2019
Member
Author
|
cherry-picked to 2.2, 2.1, 2.0, 1.1, 1.0 |
jarekr
pushed a commit
to confluentinc/kafka
that referenced
this pull request
Apr 18, 2019
* apache/trunk: KAFKA-7880:Naming worker thread by task id (apache#6275) improve some logging statements (apache#6078) KAFKA-7312: Change broker port used in testMinimumRequestTimeouts and testForceClose KAFKA-7997: Use automatic RPC generation in SaslAuthenticate KAFKA-8002; Log dir reassignment stalls if future replica has different segment base offset (apache#6346) KAFKA-3522: Add TimestampedKeyValueStore builder/runtime classes (apache#6152) HOTFIX: add igore import to streams_upgrade_test MINOR: ConsumerNetworkClient does not need to send the remaining requests when the node is not ready (apache#6264) KAFKA-7922: Return authorized operations in describe consumer group responses (KIP-430 Part-1) KAFKA-7918: Inline generic parameters Pt. III: in-memory window store (apache#6328) KAFKA-8012; Ensure partitionStates have not been removed before truncating. (apache#6333) KAFKA-8011: Fix for race condition causing concurrent modification exception (apache#6338) KAFKA-7912: Support concurrent access in InMemoryKeyValueStore (apache#6336) MINOR: Skip quota check when replica is in sync (apache#6344) HOTFIX: Change header back to http instead of https to path license header test (apache#6347) MINOR: fix release.py script (apache#6317) MINOR: Remove types from caching stores (apache#6331) MINOR: Improve logging for alter log dirs (apache#6302) MINOR: state.cleanup.delay.ms default is 600,000 ms (10 minutes). (apache#6345) MINOR: disable Streams system test for broker upgrade/downgrade (apache#6341)
pengxiaolong
pushed a commit
to pengxiaolong/kafka
that referenced
this pull request
Jun 14, 2019
…ception (apache#6338) In the RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated() and RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenDeleted() a race condition exists where the ConsumerRebalanceListener in the test modifies the list of subscribed topics when the condition for the test success is comparing the same array instance against expected values. This PR should fix this race condition by using a CopyOnWriteArrayList which guarantees safe traversal of the list even when a concurrent modification is happening. Reviewers: Guozhang Wang <wangguoz@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In the
RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated()andRegexSourceIntegrationTest#testRegexMatchesTopicsAWhenDeleted()a race condition exists where theConsumerRebalanceListenerin the test modifies the list of subscribed topics when the condition for the test success is comparing the same array instance against expected values.This PR should fix this race condition by using a
CopyOnWriteArrayListwhich guarantees safe traversal of the list even when a concurrent modification is happening.Using the
CopyOnWriteArrayListshould not impact performance negatively as the number of traversals, a result of usingArrayList.equals(), far outnumber (TestUtils.waitForCondition()checks for a successful result every100ms) the possible modifications as there will be at most one topic name added/removed during the test.For testing, I updated the
RegexSourceIntegrationTestintegration test and ran the suite of streams tests.Committer Checklist (excluded from commit message)