KAFKA-12879: Revert changes from KAFKA-12339 and instead add retry capability to KafkaBasedLog#11797
Merged
rhauch merged 29 commits intoapache:trunkfrom Mar 9, 2022
Merged
Conversation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the compatibility issue regarding KAFKA-12879 by reverting the changes to the admin client from KAFKA-12339 (#10152), and instead perform the retries within the KafkaBasedLog via a new
TopicAdmin.retryEndOffsets(..). This method will delegate to the existingTopicAdmin.endOffsets(...)and will retry on RetriableException until the retries (max retries) is hit, or until a specified timeout.This change should be backward compatible to the KAFKA-12339.
Notable changes are:
Reverted KAFKA-12339
TopicAdmin
Added new
retryEndOffsets(...)method that utilizes the newRetryUtilto list end offsets for a set of topic partitions and perform retries upon any RetriableException (including UnknownTopicOrPartitionException, TimeoutException).KafkaBasdLog
During startup calls
TopicAdmin.retryEndOffsets(...)to read end offsets for the log topic and retries if the topic is not available (or any RetriableException).RetryUtil
A general utility that takes a Callable function, the total duration to retry (may be 0), and backoff time in ms, and that calls the function at least once, and performs retries using the parameters when the function throws RetriableException (e.g. UnknownTopicOrPartitionException, TimeoutException)
Committer Checklist (excluded from commit message)