Skip to content

MINOR: Refactor GroupMetadataManagerTest#15348

Merged
dajac merged 9 commits intoapache:trunkfrom
dajac:minor-refactor-groupmetadatamanagertest
Feb 14, 2024
Merged

MINOR: Refactor GroupMetadataManagerTest#15348
dajac merged 9 commits intoapache:trunkfrom
dajac:minor-refactor-groupmetadatamanagertest

Conversation

@dajac
Copy link
Copy Markdown
Member

@dajac dajac commented Feb 9, 2024

GroupMetadataManagerTest class got a little under control. We have too many things defined in it. As a first steps, this patch extracts all the inner classes. It also extracts all the helper methods. However, the logic is not changed at all.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@dajac dajac added the KIP-848 The Next Generation of the Consumer Rebalance Protocol label Feb 9, 2024
files="(ConsumerGroupMember|GroupMetadataManager|GeneralUniformAssignmentBuilder).java"/>
<suppress checks="(NPathComplexity|MethodLength)"
files="(GroupMetadataManager|ConsumerGroupTest|GroupMetadataManagerTest|GeneralUniformAssignmentBuilder).java"/>
files="(GroupMetadataManager|ConsumerGroupTest|GroupMetadataManagerTest|GroupMetadataManagerTestContext|GeneralUniformAssignmentBuilder).java"/>
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's your take on in file suppressions vs suppressions.xml?

On a previous PR I was asked to make the supppression in file for specific methods: #15139 (comment)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was actually not aware of this. I think that it makes sense. I file https://issues.apache.org/jira/browse/KAFKA-16244 to give it a try. In this PR, I would prefer to keep the suppressions.xml exception though.


public class GroupMetadataManagerTestContext {

public static void assertResponseEquals(
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason this is here rather than the Assertions class?

Generally, I noticed there were a few static methods from the test that were moved into this class. But it seems like they don't need to be and could be in a general utils class. There are also a few static classes. Is the main idea that we don't move classes out unless they are shared?

Or we plan to do in a followup?

Copy link
Copy Markdown
Contributor

@OmniaGM OmniaGM Feb 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @jolshan this class seems to have some methods that are just assertions that maybe better to move to Assertions.java. I got that this pr isn't changing the tests themselves but moving these to the right place wouldn't change the tests and keep things clear specially that these methods didn't belong to any of the static classes before.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically, I moved methods used by at least two other classes to Assertions. I kept the others as private where they are used.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved a few more :).

Copy link
Copy Markdown
Contributor

@OmniaGM OmniaGM left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice cleanup for the tests! Just left few minor comments.

final private MockCoordinatorTimer<Void, Record> timer = new MockCoordinatorTimer<>(time);
final private LogContext logContext = new LogContext();
final private SnapshotRegistry snapshotRegistry = new SnapshotRegistry(logContext);
final private TopicPartition groupMetadataTopicPartition = new TopicPartition("topic", 0);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

groupMetadataTopicPartition doesn't seem to be used anywhere

return timeouts;
}

public MockCoordinatorTimer.ScheduledTimeout<Void, Record> assertSessionTimeout(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The return value here doesn't seem to be used by any of the tests can't we just return void?

}

public List<ListGroupsResponseData.ListedGroup> sendListGroups(List<String> statesFilter, List<String> typesFilter) {
Set<String> statesFilterSet = statesFilter.stream().collect(Collectors.toSet());
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't these be converted to be new HashSet(statesFilter) instead of streaming and collecting


public List<ListGroupsResponseData.ListedGroup> sendListGroups(List<String> statesFilter, List<String> typesFilter) {
Set<String> statesFilterSet = statesFilter.stream().collect(Collectors.toSet());
Set<String> typesFilterSet = typesFilter.stream().collect(Collectors.toSet());
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

if (assignment == null) return null;

Map<Uuid, Set<Integer>> assignmentMap = new HashMap<>();
assignment.forEach(topicPartitions -> {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there're few places in the pr where statement lambda can be converted to expression lambda instead. would be nice cleanup

@dajac
Copy link
Copy Markdown
Member Author

dajac commented Feb 12, 2024

@OmniaGM @jolshan Thanks for your comments. I have addressed them.

@dajac dajac requested review from OmniaGM and jolshan February 12, 2024 13:41
Copy link
Copy Markdown
Contributor

@OmniaGM OmniaGM left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update. Just some small cleanups but not a blocker for the PR.

if (assignment == null) return null;

Map<Uuid, Set<Integer>> assignmentMap = new HashMap<>();
assignment.forEach(topicPartitions -> {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This also can be expression lambda

List<ConsumerGroupCurrentMemberAssignmentValue.TopicPartitions> assignment
) {
Map<Uuid, Set<Integer>> assignmentMap = new HashMap<>();
assignment.forEach(topicPartitions -> {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same could be an expression lambda

List<Record> records = new ArrayList<>();

// Add subscription records for members.
members.forEach((memberId, member) -> {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

// Add subscription metadata.
if (subscriptionMetadata == null) {
subscriptionMetadata = new HashMap<>();
members.forEach((memberId, member) -> {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

records.add(RecordHelpers.newGroupEpochRecord(groupId, groupEpoch));

// Add target assignment records.
assignments.forEach((memberId, assignment) -> {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

records.add(RecordHelpers.newTargetAssignmentEpochRecord(groupId, assignmentEpoch));

// Add current assignment records for members.
members.forEach((memberId, member) -> {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

Copy link
Copy Markdown
Member

@jolshan jolshan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks David!

@dajac dajac merged commit d24abe0 into apache:trunk Feb 14, 2024
@dajac dajac deleted the minor-refactor-groupmetadatamanagertest branch February 14, 2024 07:29
yyu1993 pushed a commit to yyu1993/kafka that referenced this pull request Feb 15, 2024
`GroupMetadataManagerTest` class got a little under control. We have too many things defined in it. As a first steps, this patch extracts all the inner classes. It also extracts all the helper methods. However, the logic is not changed at all.

Reviewers: Omnia Ibrahim <o.g.h.ibrahim@gmail.com>, Justine Olshan <jolshan@confluent.io>
clolov pushed a commit to clolov/kafka that referenced this pull request Apr 5, 2024
`GroupMetadataManagerTest` class got a little under control. We have too many things defined in it. As a first steps, this patch extracts all the inner classes. It also extracts all the helper methods. However, the logic is not changed at all.

Reviewers: Omnia Ibrahim <o.g.h.ibrahim@gmail.com>, Justine Olshan <jolshan@confluent.io>
Phuc-Hong-Tran pushed a commit to Phuc-Hong-Tran/kafka that referenced this pull request Jun 6, 2024
`GroupMetadataManagerTest` class got a little under control. We have too many things defined in it. As a first steps, this patch extracts all the inner classes. It also extracts all the helper methods. However, the logic is not changed at all.

Reviewers: Omnia Ibrahim <o.g.h.ibrahim@gmail.com>, Justine Olshan <jolshan@confluent.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

KIP-848 The Next Generation of the Consumer Rebalance Protocol

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants