KAFKA-13072: Make RemoveMembersFromConsumerGroupHandler unmap for COORDINATOR_NOT_AVAILABLE error by showuon · Pull Request #11035 · apache/kafka

showuon · 2021-07-13T09:42:25Z

refactor RemoveMembersFromConsumerGroupHandler and tests. Also, put COORDINATOR_NOT_AVAILABLE as unmap retry.

This is the old handle response logic. FYR:

void handleResponse(AbstractResponse abstractResponse) {
      final LeaveGroupResponse response = (LeaveGroupResponse) abstractResponse;

      // If coordinator changed since we fetched it, retry
      // note here: we'll collect all errors in group error and member errors, to check if if we need to retry
      if (ConsumerGroupOperationContext.hasCoordinatorMoved(response)) {
          Call call = getRemoveMembersFromGroupCall(context, members);
          rescheduleFindCoordinatorTask(context, () -> call, this);
          return;
      }

      if (handleGroupRequestError(response.topLevelError(), context.future()))
          return;

      final Map<MemberIdentity, Errors> memberErrors = new HashMap<>();
      for (MemberResponse memberResponse : response.memberResponses()) {
          memberErrors.put(new MemberIdentity()
                               .setMemberId(memberResponse.memberId())
                               .setGroupInstanceId(memberResponse.groupInstanceId()),
                           Errors.forCode(memberResponse.errorCode()));
      }
      context.future().complete(memberErrors);
  }

Committer Checklist (excluded from commit message)

Verify design and implementation
Verify test coverage and CI build status
Verify documentation (including upgrade notes)

showuon · 2021-07-13T09:42:46Z

@dajac , please take a look. Thanks.

dajac · 2021-07-13T15:22:16Z

+                String memberId = memberResponse.memberId();
+
+                if (memberError != Errors.NONE) {
+                    handleMemberError(groupId, memberId, memberError, groupsToUnmap, groupsToRetry);


Is this really necessary? It seems that group level errors as always retuned as top level error in the response. Do you confirm?

Thanks for the question. I checked broker code, and confirmed that all member response only contain member level errors, not group level or coordinator level. I remove it. Thanks.

dajac

@showuon Thanks. I left a few more comments.

dajac · 2021-07-14T19:46:55Z

        }
-        return new ApiResult<>(completed, failed, unmapped);
+
+        if (groupsToUnmap.isEmpty() && groupsToRetry.isEmpty()) {


This does not seem necessary as we always expect the top level error. Would it make sense to handle it like we did here: https://github.com/apache/kafka/pull/11019/files#diff-e7eafbafe0b75099d0c8b4083c03c653d57245ef7b0fcfae7b9ccd258a9024e3R117?

Agree! Updated!

dajac · 2021-07-14T19:47:30Z

+
                memberErrors.put(new MemberIdentity()
-                                     .setMemberId(memberResponse.memberId())
+                                     .setMemberId(memberId)


nit: We could revert this change as it does not bring much.

dajac · 2021-07-14T19:47:50Z

+                                     .setMemberId(memberId)
                                     .setGroupInstanceId(memberResponse.groupInstanceId()),
-                                 Errors.forCode(memberResponse.errorCode()));
+                    memberError);


nit: We could revert this change as it does not bring much and re-align like it was before.

dajac · 2021-07-14T19:48:33Z

    }

-}
+}


nit: Could we add an empty line back?

dajac · 2021-07-14T19:49:03Z

+                final String unexpectedErrorMsg =
+                    String.format("`LeaveGroup` request for group id %s failed due to unexpected error %s", groupId.idValue, error);
+                log.error(unexpectedErrorMsg);
+                failed.put(groupId, error.exception(unexpectedErrorMsg));


nit: As said in the other PR, this is a good idea but I would only do it if we do it for all exceptions.

dajac · 2021-07-14T19:50:08Z

+            MemberResponse memberResponse = new MemberResponse()
+                .setGroupInstanceId(groupId)
+                .setErrorCode(Errors.COORDINATOR_LOAD_IN_PROGRESS.code());


I guess that we could remove this test as it is not possible to have COORDINATOR_LOAD_IN_PROGRESS as an error for a member, right?

Right! Removed. Thanks.

dajac

LGTM, thanks.

dajac · 2021-07-15T12:34:51Z

I've re-triggered Jenkins.

showuon · 2021-07-15T12:42:43Z

Thank you. it have been running for whole day. :)

dajac · 2021-07-15T14:55:22Z

Failures are not related:

Build / JDK 16 and Scala 2.13 / testCloseDuringRebalance() – kafka.api.ConsumerBounceTest
4s
Build / JDK 16 and Scala 2.13 / testCloseDuringRebalance() – kafka.api.ConsumerBounceTest
10s
Build / JDK 11 and Scala 2.13 / testSendOffsetsToTransactionTimeout() – kafka.api.TransactionsTest
9s
Build / JDK 11 and Scala 2.13 / testSendOffsetsToTransactionTimeout() – kafka.api.TransactionsTest
13s

…RDINATOR_NOT_AVAILABLE error (#11035) This patch improve the error handling in `RemoveMembersFromConsumerGroupHandler` and ensures that `COORDINATOR_NOT_AVAILABLE` is unmapped in order to look up the coordinator again. Reviewers: David Jacot <djacot@confluent.io>

dajac · 2021-07-15T16:05:18Z

Merged to trunk and 3.0. cc @kkonstantine

…RDINATOR_NOT_AVAILABLE error (apache#11035) This patch improve the error handling in `RemoveMembersFromConsumerGroupHandler` and ensures that `COORDINATOR_NOT_AVAILABLE` is unmapped in order to look up the coordinator again. Reviewers: David Jacot <djacot@confluent.io>

KAFKA-13072: refactor RemoveMembersFromConsumerGroupHandler and tests

b4a3bfa

showuon mentioned this pull request Jul 13, 2021

KAFKA-13033: COORDINATOR_NOT_AVAILABLE should be unmapped #10973

Closed

3 tasks

dajac reviewed Jul 13, 2021

View reviewed changes

KAFKA-13072: refactor code

b7acfcd

showuon force-pushed the KAFKA-13072 branch from ba15b6e to b7acfcd Compare July 14, 2021 07:17

KAFKA-13072: refactor

a4e7590

dajac reviewed Jul 14, 2021

View reviewed changes

KAFKA-13072: remove unneeded test and refactor

0fb04e9

showuon force-pushed the KAFKA-13072 branch from 3aebe51 to 0fb04e9 Compare July 15, 2021 02:55

showuon changed the title ~~KAFKA-13072: refactor RemoveMembersFromConsumerGroupHandler and tests~~ KAFKA-13072: Make RemoveMembersFromConsumerGroupHandler unmap for COORDINATOR_NOT_AVAILABLE error Jul 15, 2021

dajac approved these changes Jul 15, 2021

View reviewed changes

dajac merged commit 921a342 into apache:trunk Jul 15, 2021

                   }
-              }
+              }

                
                    No newline at end of file

Conversation

showuon commented Jul 13, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Committer Checklist (excluded from commit message)

Uh oh!

showuon commented Jul 13, 2021

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dajac left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dajac left a comment

Choose a reason for hiding this comment

Uh oh!

dajac commented Jul 15, 2021

Uh oh!

showuon commented Jul 15, 2021

Uh oh!

dajac commented Jul 15, 2021

Uh oh!

dajac commented Jul 15, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

showuon commented Jul 13, 2021 •

edited

Loading