KAFKA-10340: Backport proactively close producer when cancelling source tasks#10238
Closed
rhauch wants to merge 1 commit intoapache:2.8from
Closed
KAFKA-10340: Backport proactively close producer when cancelling source tasks#10238rhauch wants to merge 1 commit intoapache:2.8from
rhauch wants to merge 1 commit intoapache:2.8from
Conversation
…pache#10016) Close the producer in `WorkerSourceTask` when the latter is cancelled. If the broker do not autocreate the topic, and the connector is not configured to create topics written by the source connector, then the `WorkerSourceTask` main thread will block forever until the topic is created, and will not stop if cancelled or scheduled for shutdown by the worker. Expanded an existing unit test for the WorkerSourceTask class to ensure that the producer is closed when the task is abandoned, and added a new integration test that guarantees that tasks are still shut down even when their producers are trying to write to topics that do not exist. Author: Chris Egerton <chrise@confluent.io> Reviewed: Greg Harris <gregh@confluent.io>, Randall Hauch <rhauch@gmail.com>
rhauch
changed the title
Contributor
Author
|
Closed without merging. Instead, I cherry-picked the commit from #10016 to the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
NOTE: This is targeted to the
2.8branch (currently frozen), but includes the same commit cherry-picked fromtrunkand originating in #10016 for KAFKA-10340.This should be merged only after this issue is approved as a blocker for 2.8.0, or after 2.8.0 is released. When it is merged, add "2.8.0" or "2.8.1" to the fix version for KAFKA-10340.
Close the producer in
WorkerSourceTaskwhen the latter is cancelled. If the broker do not autocreate the topic, and the connector is not configured to create topics written by the source connector, then theWorkerSourceTaskmain thread will block forever until the topic is created, and will not stop if cancelled or scheduled for shutdown by the worker.Expanded an existing unit test for the WorkerSourceTask class to ensure that the producer is closed when the task is abandoned, and added a new integration test that guarantees that tasks are still shut down even when their producers are trying to write to topics that do not exist.