Skip to content

MINOR: Better messaging for invalid fetch response#6427

Merged
hachikuji merged 2 commits intoapache:trunkfrom
jsancio:kafka-7565
Mar 12, 2019
Merged

MINOR: Better messaging for invalid fetch response#6427
hachikuji merged 2 commits intoapache:trunkfrom
jsancio:kafka-7565

Conversation

@jsancio
Copy link
Copy Markdown
Member

@jsancio jsancio commented Mar 11, 2019

Users have reported (KAFKA-7565) that when consumer poll wake up is used,
it is possible to receive fetch responses that don't match the copied topic
partitions collection for the session when the fetch request was created.

This commit improves the error handling here by throwing an
IllegalStateException instead of a NullPointerException. And by
generating a message for the exception that includes a bit of more
information.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

Users have reported that when consumer poll wake up is used, it is
possible to receive fetch responses that don't match the copied topic
partitions collection for the session when the fetch request was created.

This commit improves the error handling here by throwing an
IllegalStateException instead of a NullPointerException. And by
generating a message for the exception that includes a bit of more
information.
@jsancio
Copy link
Copy Markdown
Member Author

jsancio commented Mar 11, 2019

cc @hachikuji @cmccabe

Comment thread clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java Outdated
Copy link
Copy Markdown
Contributor

@hachikuji hachikuji left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I'd suggest changing this to a MINOR PR since we are not addressing the root cause of KAFKA-7565.

@jsancio jsancio changed the title KAFKA-7565: Better messaging for invalid fetch response MINOR PR: Better messaging for invalid fetch response Mar 12, 2019
@jsancio
Copy link
Copy Markdown
Member Author

jsancio commented Mar 12, 2019

LGTM. I'd suggest changing this to a MINOR PR since we are not addressing the root cause of KAFKA-7565.

Done.

@hachikuji hachikuji changed the title MINOR PR: Better messaging for invalid fetch response MINOR: Better messaging for invalid fetch response Mar 12, 2019
@hachikuji hachikuji merged commit fd79dd0 into apache:trunk Mar 12, 2019
@jsancio jsancio deleted the kafka-7565 branch March 12, 2019 22:37
if (data.metadata().isFull()) {
message = MessageFormatter.arrayFormat(
"Response for missing full request partition: partition={}; metadata={}",
new Object[]{partition, data.metadata()}).getMessage();
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason why we didn't use String.format here?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No technical reason. I see that String.format is used in a few places already. Let me know if you would like me to change this.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ijuma, Actually, I now remember why I did it. MessageFormatter supports Java arrays while String.format does not.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why would you want arrays instead of varargs though?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

String.format("array=%s", new Object[]{1,2,3})
array=[I@76f78e3a

MessageFormatter.arrayFormat("array={}", new Object[]{new Object[]{1,2,3}}).getMessage()
array=[1,2,3]

Right?

jarekr pushed a commit to confluentinc/kafka that referenced this pull request Apr 18, 2019
* apache/trunk:
  MINOR: Retain public constructors of classes from public API (apache#6455)
  KAFKA-8118; Ensure ZK clients are closed in tests, fix verification (apache#6456)
  KAFKA-7813: JmxTool throws NPE when --object-name is omitted
  KAFKA-8114: Wait for SCRAM credential propagation in DelegationTokenEndToEndAuthorizationTest (apache#6452)
  KAFKA-8111; Set min and max versions for Metadata requests (apache#6451)
  KAFKA-7855: Kafka Streams Maven Archetype quickstart fails to compile out of the box (apache#6194)
  MINOR: Update code to not use deprecated methods (apache#6434)
  MINOR: Update Trogdor ConnectionStressWorker status at the end of execution (apache#6445)
  KAFKA-8091; Use commitSync to check connection failure in listener update test (apache#6450)
  KAFKA-7027: Add an overload build method in scala (apache#6373)
  MINOR: Fix typos in LogValidator (apache#6449)
  KAFKA-7502: Cleanup KTable materialization logic in a single place (apache#6174)
  KAFKA-7730; Limit number of active connections per listener in brokers (KIP-402)
  KAFKA-8091; Remove unsafe produce from dynamic listener update test (apache#6443)
  MINOR: Fix JavaDocs warnings (apache#6435)
  MINOR: Better messaging for invalid fetch response (apache#6427)
  MINOR: Use Java 8 lambdas in KStreamImplTest (apache#6430)
pengxiaolong pushed a commit to pengxiaolong/kafka that referenced this pull request Jun 14, 2019
Users have reported (KAFKA-7565) that when consumer poll wake up is used,
it is possible to receive fetch responses that don't match the copied topic
partitions collection for the session when the fetch request was created.

This commit improves the error handling here by throwing an
IllegalStateException instead of a NullPointerException. And by
generating a message for the exception that includes a bit of more
information.

Reviewers: Jason Gustafson <jason@confluent.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants