MINOR: Improve EOS example exception handling#8052
Conversation
There was a problem hiding this comment.
I don't think this is the only case that we can abort a transaction. This only handles offset commit failures, but what about send failures?
f6eecf1 to
8291e68
Compare
hachikuji
left a comment
There was a problem hiding this comment.
Thanks for updating this. Left a few comments.
There was a problem hiding this comment.
Why is InvalidOffsetException fatal?
There was a problem hiding this comment.
I guess it could be retriable.
There was a problem hiding this comment.
Could we do "group mode" only in this example? The example doesn't really extend to multiple instances otherwise.
There was a problem hiding this comment.
Not sure I follow, we could manually compute the partitions to assign to for the standalone mode?
There was a problem hiding this comment.
Hmm.. Is it actually safe to abort following a TimeoutException? I think this would cause an illegal state error in the producer. To handle this correctly, the application should retry the commit.
There was a problem hiding this comment.
Or we shouldn't handle the timeout, as we always rely on max_block to ensure the request is successful?
0996e26 to
e84165a
Compare
There was a problem hiding this comment.
For the purpose of understanding EOS, the main exceptions that are worth calling out are ProducerFencedException and FencedInstanceIdException. I would suggest we write the example like this:
try {
...
producer.commitTransaction;
} catch (ProducerFencedException e) {
throw KafkaException("The transactional.id $transactionalId has been claimed by another process");
} catch (FencedInstanceIdException e) {
throw KafkaException("The group.instance.id $instanceId has been claimed by another process");
} catch (KafkaException e) {
// If we have not been fenced, try to abort the transaction and continue. This will raise immediately
// if the producer has hit a fatal error.
producer.abortTransaction();
}eca8adb to
e8fc78f
Compare
|
retest this please |
e8fc78f to
b1905ec
Compare
hachikuji
left a comment
There was a problem hiding this comment.
LGTM. Thanks for simplifying this example.
|
retest this please |
|
ok to test |
The current EOS example mixes fatal and non-fatal error handling. This patch fixes this problem and simplifies the example. Reviewers: Jason Gustafson <jason@confluent.io>
|
Cherry-picked to 2.5 |
* apache-github/trunk: (23 commits) KAFKA-9530; Fix flaky test `testDescribeGroupWithShortInitializationTimeout` (apache#8154) HOTFIX: fix NPE in Kafka Streams IQ (apache#8158) MINOR: set scala version automatically based on gradle.properties KAFKA-9577; SaslClientAuthenticator incorrectly negotiates SASL_HANDSHAKE version (apache#8142) KAFKA-9441: Add internal TransactionManager (apache#8105) MINOR: Document endpoints for connector topic tracking (KIP-558) MINOR: Standby task commit needed when offsets updated (apache#8146) KAFKA-9206; Throw KafkaException on CORRUPT_MESSAGE error in Fetch response (apache#8111) MINOR: Remove unwanted regexReplace on tests/kafkatest/__init__.py KAFKA-9586: Fix errored json filename in ops documentation KAFKA-9575: Mention ZooKeeper 3.5.7 upgrade KAFKA-9481: Graceful handling TaskMigrated and TaskCorrupted (apache#8058) HOTFIX: don't try to remove uninitialized changelogs from assignment & don't prematurely mark task closed (apache#8140) MINOR: Fix javadoc at org.apache.kafka.clients.producer.KafkaProducer.InterceptorCallback#onCompletion (apache#7337) MINOR: Improve EOS example exception handling (apache#8052) MINOR: Fix a number of warnings in clients test (apache#8073) MINOR: Update shell scripts to support z/OS system (apache#7913) MINOR: Wording fix in Streams DSL docs (apache#5692) MINOR: Add missing @test annotation to MetadataTest#testMetadataMerge (apache#8141) KAFKA-9533: ValueTransform forwards `null` values (apache#8108) ...
…etrics-common * confluent/master: (76 commits) KAFKA-9530; Fix flaky test `testDescribeGroupWithShortInitializationTimeout` (apache#8154) HOTFIX: fix NPE in Kafka Streams IQ (apache#8158) MINOR: set scala version automatically based on gradle.properties KAFKA-9577; SaslClientAuthenticator incorrectly negotiates SASL_HANDSHAKE version (apache#8142) KAFKA-9441: Add internal TransactionManager (apache#8105) MINOR: Document endpoints for connector topic tracking (KIP-558) MINOR: Standby task commit needed when offsets updated (apache#8146) Changes to migrate to Artifactory (#263) KAFKA-9206; Throw KafkaException on CORRUPT_MESSAGE error in Fetch response (apache#8111) MINOR: Remove unwanted regexReplace on tests/kafkatest/__init__.py KAFKA-9586: Fix errored json filename in ops documentation KAFKA-9575: Mention ZooKeeper 3.5.7 upgrade KAFKA-9481: Graceful handling TaskMigrated and TaskCorrupted (apache#8058) HOTFIX: don't try to remove uninitialized changelogs from assignment & don't prematurely mark task closed (apache#8140) MINOR: Fix javadoc at org.apache.kafka.clients.producer.KafkaProducer.InterceptorCallback#onCompletion (apache#7337) MINOR: Improve EOS example exception handling (apache#8052) MINOR: Fix a number of warnings in clients test (apache#8073) MINOR: Update shell scripts to support z/OS system (apache#7913) MINOR: Wording fix in Streams DSL docs (apache#5692) MINOR: Add missing @test annotation to MetadataTest#testMetadataMerge (apache#8141) ...
The current EOS example is over-complicating the exception handling by mixing non fatal and fatal ones. This cleanup is trying to make the code readability better. Will rebase after #8051 is merged
Committer Checklist (excluded from commit message)