Skip to content

KAFKA-10435; Fetch protocol changes for KIP-595#9275

Merged
hachikuji merged 3 commits intoapache:trunkfrom
hachikuji:KAFKA-10435
Sep 15, 2020
Merged

KAFKA-10435; Fetch protocol changes for KIP-595#9275
hachikuji merged 3 commits intoapache:trunkfrom
hachikuji:KAFKA-10435

Conversation

@hachikuji
Copy link
Copy Markdown
Contributor

This patch bumps the Fetch protocol as specified by KIP-595: https://cwiki.apache.org/confluence/display/KAFKA/KIP-595%3A+A+Raft+Protocol+for+the+Metadata+Quorum. The main differences are the following:

  • Truncation detection
  • Leader discovery through the response
  • Flexible version support

The most notable change is truncation detection. This patch adds logic in the request handling path to detect truncation, but it does not change the replica fetchers to make use of this capability. We are planning to do this separately.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@hachikuji
Copy link
Copy Markdown
Contributor Author

Note this patch includes #9277. I will rebase once it is merged.

Copy link
Copy Markdown
Contributor

@rajinisivaram rajinisivaram left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hachikuji Thanks for the PR. Left a few minor comments/questions, apart from that LGTM.

Comment thread clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java Outdated
Comment thread core/src/test/scala/unit/kafka/server/FetchRequestTest.scala Outdated
Comment thread core/src/main/scala/kafka/server/epoch/LeaderEpochFileCache.scala
Copy link
Copy Markdown
Contributor

@rajinisivaram rajinisivaram left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hachikuji Thanks for the PR. Left a few minor comments/questions, apart from that LGTM.

Copy link
Copy Markdown
Contributor

@rajinisivaram rajinisivaram left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hachikuji Thanks for the update, looks good. Left one question because I wasn't sure. Apart from that LGTM

Comment thread core/src/main/scala/kafka/server/epoch/LeaderEpochFileCache.scala Outdated
Copy link
Copy Markdown
Contributor

@rajinisivaram rajinisivaram left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hachikuji Thanks for the update, LGTM

@hachikuji hachikuji merged commit 634c917 into apache:trunk Sep 15, 2020
// Introduced feature versioning support (KIP-584)
KAFKA_2_7_IV0
KAFKA_2_7_IV0,
// Bup Fetch protocol for Raft protocol (KIP-595)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe you can fix this typo in your next PR.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops! Will do.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants