KAFKA-7519 Clear pending transaction state when expiration fails#5820
Conversation
|
retest this please |
BirdHowl
force-pushed
the
kafka-7519-pending-state-on-expired-transactions
branch
from
7c0f7f0 to
f1ab055
Compare
|
Decided to change the tests to always check that metadata we find isn't in a pending state after transactional ID expiration. My reasoning is that since the transactional id cleanup task is a background activity, I'd expect that it should only leave metadata in a pending state for a very short period, to avoid disrupting normal requests. Limiting that to the duration of a single instance of the cleanup task is a good restriction if we can follow it; it also avoids cases where we have to wait for the period of the cleanup task (1 hour by default is pretty bad) in addition to my original problem. |
hachikuji
left a comment
hachikuji
left a comment
There was a problem hiding this comment.
LGTM. I'll push a minor formatting fix before merging. Thanks for the fix!
hachikuji
left a comment
hachikuji
left a comment
There was a problem hiding this comment.
LGTM. I'll push a minor formatting fix before merging. Thanks for the fix!
hachikuji
left a comment
hachikuji
left a comment
There was a problem hiding this comment.
LGTM. I'll push a minor formatting tweak before merging. Thanks for the fix!
Make sure that the transaction state is properly cleared when the `transactionalId-expiration` task fails. Operations on that transactional id would otherwise return a `CONCURRENT_TRANSACTIONS` error and appear "untouchable" to transaction state changes, preventing transactional producers from operating until a broker restart or transaction coordinator change. Unit tested by verifying that having the `transactionalId-expiration` task won't leave the transaction metadata in a pending state if the replica manager returns an error. Reviewers: Jason Gustafson <jason@confluent.io>
Make sure that the transaction state is properly cleared when the `transactionalId-expiration` task fails. Operations on that transactional id would otherwise return a `CONCURRENT_TRANSACTIONS` error and appear "untouchable" to transaction state changes, preventing transactional producers from operating until a broker restart or transaction coordinator change. Unit tested by verifying that having the `transactionalId-expiration` task won't leave the transaction metadata in a pending state if the replica manager returns an error. Reviewers: Jason Gustafson <jason@confluent.io>
|
Jason asked me to merge this. JDK 8 tests passed, JDK 11 test failures unrelated. Merged to trunk, 2.1 and 2.0 branches. |
…che#5820) Make sure that the transaction state is properly cleared when the `transactionalId-expiration` task fails. Operations on that transactional id would otherwise return a `CONCURRENT_TRANSACTIONS` error and appear "untouchable" to transaction state changes, preventing transactional producers from operating until a broker restart or transaction coordinator change. Unit tested by verifying that having the `transactionalId-expiration` task won't leave the transaction metadata in a pending state if the replica manager returns an error. Reviewers: Jason Gustafson <jason@confluent.io>
3 participants
Description:
Make sure that the transaction state is properly cleared when the
transactionalId-expirationtask fails. Operations on that transactional id would otherwise return aCONCURRENT_TRANSACTIONSerror and appear "untouchable" to transaction state changes, preventing transactional producers from operating until a broker restart or transaction coordinator change.Testing:
Unit tested by verifying that having the
transactionalId-exprationtask won't leave the transaction metadata in a pending state if the replica manager returns an error.Committer Checklist (excluded from commit message)