KAFKA-6145: Add unit tests to verify fix of bug KAFKA-9173#8689
Merged
vvcephei merged 2 commits intoapache:trunkfrom May 19, 2020
Merged
KAFKA-6145: Add unit tests to verify fix of bug KAFKA-9173#8689vvcephei merged 2 commits intoapache:trunkfrom
vvcephei merged 2 commits intoapache:trunkfrom
Conversation
Unit tests - shouldAssignActiveStatefulTasksEvenlyOverClientsAndStreamThreadsWithMoreStreamThreadsThanTasks() - shouldAssignWarmUpTasksIfStatefulActiveTasksBalancedOverStreamThreadsButNotOverClients() - shouldEvenlyAssignActiveStatefulTasksIfClientsAreWarmedUpToBalanceTaskOverClients() verify that bug KAFKA-9173 is fixed with the new HighAvailabilityTaskAssignor. shouldAssignActiveStatefulTasksEvenlyOverClientsAndStreamThreadsWithMoreStreamThreadsThanTasks() ensures that tasks are evenly assigned over clients when all overprovisioned clients join simultaneously. shouldAssignWarmUpTasksIfStatefulActiveTasksBalancedOverStreamThreadsButNotOverClients() ensures that warm-up tasks are assigned to two new clients that join the group although the assignment is already balanced over stream threads. shouldEvenlyAssignActiveStatefulTasksIfClientsAreWarmedUpToBalanceTaskOverClients() ensures that stateful active tasks are balanced over previous and warmed-up client although it the previous assignment is balanced over stream threads.
cadonna
commented
May 18, 2020
|
|
||
| @Test | ||
| public void shouldAssignActiveStatefulTasksEvenlyOverClientsWithLessClientsThanTasks() { | ||
| public void shouldAssignActiveStatefulTasksEvenlyOverClientsWithMoreClientsThanTasks() { |
Member
Author
There was a problem hiding this comment.
This name seemed not correct.
Member
Author
|
Call for review: @vvcephei @ableegoldman |
vvcephei
reviewed
May 19, 2020
|
|
||
| @Test | ||
| public void shouldAssignActiveStatefulTasksEvenlyOverClientsWithLessClientsThanTasks() { | ||
| public void shouldAssignActiveStatefulTasksEvenlyOverClientsWithMoreClientsThanTasks() { |
| ); | ||
|
|
||
| assertThat(unstable, is(false)); | ||
| assertValidAssignment(0, allTaskIds.size() / 3 + 1, allTaskIds, emptySet(), clientStates, new StringBuilder()); |
Contributor
There was a problem hiding this comment.
If we expect no warmups, we can assert it here with:
Suggested change
| assertValidAssignment(0, allTaskIds.size() / 3 + 1, allTaskIds, emptySet(), clientStates, new StringBuilder()); | |
| assertValidAssignment(0, allTaskIds, emptySet(), clientStates, new StringBuilder()); |
Contributor
|
test this please |
Contributor
|
test this please |
2 similar comments
Contributor
|
test this please |
Contributor
|
test this please |
Contributor
|
retest this please |
Contributor
|
Test failures were unrelated: and 3x: org.apache.kafka.streams.integration.EosBetaUpgradeIntegrationTest.shouldUpgradeFromEosAlphaToEosBeta[true] |
Contributor
|
Thanks, @cadonna ! |
Kvicii
pushed a commit
to Kvicii/kafka
that referenced
this pull request
May 21, 2020
* 'trunk' of github.com:apache/kafka: MINOR: Increase gradle daemon’s heap size to 2g (apache#8700) KAFKA-9603: Do not turn on bulk loading for segmented stores on stand-by tasks (apache#8661) KAFKA-9859 / kafka-streams-application-reset tool doesn't take into account topics generated by KTable foreign key join operation (apache#8671) MINOR: Fix redundant typos in comments and javadocs (apache#8693) KAFKA-10010: Should make state store registration idempotent (apache#8681) KAFKA-10011: Remove task id from lockedTaskDirectories during handleLostAll (apache#8682) KAFKA-9992: Eliminate JavaConverters in EmbeddedKafkaCluster (apache#8673) KAFKA-6145: Add unit tests to verify fix of bug KAFKA-9173 (apache#8689) MINOR: Update stream documentation (apache#8622) MINOR: Small fixes in the documentation (apache#8623)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Unit tests
shouldAssignActiveStatefulTasksEvenlyOverClientsAndStreamThreadsWithMoreStreamThreadsThanTasks()shouldAssignWarmUpTasksIfStatefulActiveTasksBalancedOverStreamThreadsButNotOverClients()shouldEvenlyAssignActiveStatefulTasksIfClientsAreWarmedUpToBalanceTaskOverClients()verify that bug KAFKA-9173 is fixed with the new
HighAvailabilityTaskAssignor.shouldAssignActiveStatefulTasksEvenlyOverClientsAndStreamThreadsWithMoreStreamThreadsThanTasks()ensures that tasks are evenly assigned over clients when all overprovisioned clients join
simultaneously.
shouldAssignWarmUpTasksIfStatefulActiveTasksBalancedOverStreamThreadsButNotOverClients()ensures that warm-up tasks are assigned to two new clients that join the group
although the assignment is already balanced over stream threads.
shouldEvenlyAssignActiveStatefulTasksIfClientsAreWarmedUpToBalanceTaskOverClients()ensures that stateful active tasks are balanced over previous and warmed-up client
although it the previous assignment is balanced over stream threads.
Committer Checklist (excluded from commit message)