Skip to content

KAFKA-12257: Consumer mishandles topics deleted and recreated with the same name (for 3.0)#10999

Closed
jolshan wants to merge 2 commits intoapache:3.0from
jolshan:KAFKA-12257
Closed

KAFKA-12257: Consumer mishandles topics deleted and recreated with the same name (for 3.0)#10999
jolshan wants to merge 2 commits intoapache:3.0from
jolshan:KAFKA-12257

Conversation

@jolshan
Copy link
Copy Markdown
Member

@jolshan jolshan commented Jul 8, 2021

Opening a version of #10952 for the 3.0 branch

Store topic ID info in consumer metadata. We will always take the topic ID from the latest metadata response and remove any topic IDs from the cache if the metadata response did not return a topic ID for the topic.

With the addition of topic IDs, when we encounter a new topic ID (recreated topic) we can choose to get the topic's metadata even if the epoch is lower than the deleted topic.

The idea is that when we update from no topic IDs to using topic IDs, we will not count the topic as new (It could be the same topic but with a new ID). We will only take the update if the topic ID changed.

Added tests for this scenario as well as some tests for storing the topic IDs. Also added tests for topic IDs in metadata cache.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@jolshan
Copy link
Copy Markdown
Member Author

jolshan commented Jul 8, 2021

I realized I can change the target of the original. I'll open a new PR for the revised trunk version.

@jolshan jolshan closed this Jul 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant