Skip to content

KAFKA-12879: Revert changes from KAFKA-12339 and instead add retry capability to KafkaBasedLog#11797

Merged
rhauch merged 29 commits intoapache:trunkfrom
philipnee:kafka-12879-connect
Mar 9, 2022
Merged

KAFKA-12879: Revert changes from KAFKA-12339 and instead add retry capability to KafkaBasedLog#11797
rhauch merged 29 commits intoapache:trunkfrom
philipnee:kafka-12879-connect

Conversation

@philipnee
Copy link
Copy Markdown
Contributor

@philipnee philipnee commented Feb 23, 2022

Summary
Fixes the compatibility issue regarding KAFKA-12879 by reverting the changes to the admin client from KAFKA-12339 (#10152), and instead perform the retries within the KafkaBasedLog via a new TopicAdmin.retryEndOffsets(..). This method will delegate to the existing TopicAdmin.endOffsets(...) and will retry on RetriableException until the retries (max retries) is hit, or until a specified timeout.

This change should be backward compatible to the KAFKA-12339.

Notable changes are:
Reverted KAFKA-12339

TopicAdmin
Added new retryEndOffsets(...) method that utilizes the new RetryUtil to list end offsets for a set of topic partitions and perform retries upon any RetriableException (including UnknownTopicOrPartitionException, TimeoutException).

KafkaBasdLog
During startup calls TopicAdmin.retryEndOffsets(...) to read end offsets for the log topic and retries if the topic is not available (or any RetriableException).

RetryUtil
A general utility that takes a Callable function, the total duration to retry (may be 0), and backoff time in ms, and that calls the function at least once, and performs retries using the parameters when the function throws RetriableException (e.g. UnknownTopicOrPartitionException, TimeoutException)

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants