[feature][txn] Fix individual ack batch message with transaction abort redevlier duplicate messages #14327
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… will redeliver Duplicate bit sit
congbobo184
requested review from
315157973,
codelipenghui,
eolivelli,
gaoran10 and
merlimat
|
@congbobo184:Thanks for your contribution. For this PR, do we need to update docs? |
congbobo184
added
doc-not-needed
|
@congbobo184:Thanks for providing doc info! |
congbobo184
added
doc-not-needed
liangyepianzhou
approved these changes
Feb 23, 2022
|
The pr had no activity for 30 days, mark with Stale label. |
Jason918
approved these changes
Mar 26, 2022
managed-ledger/src/main/java/org/apache/bookkeeper/mledger/util/PositionAckSetUtil.java
Outdated
Show resolved
Hide resolved
added 3 commits
April 2, 2022 09:45
…tion_individual_ack_batch_message_abort_problem
…tion_individual_ack_batch_message_abort_problem
congbobo184
changed the title
|
The pr had no activity for 30 days, mark with Stale label. |
Contributor
|
@congbobo184 Should we close this PR? |
Contributor
Author
|
@Jason918 don't close this PR, I will deal with it |
15 tasks
added 2 commits
February 16, 2023 21:48
Member
|
Isn't there a problem of message loss with this PR? |
Contributor
|
We should carefully to cherry-pick this fix to release branch. |
Member
|
@thetumbled I will do testing to check the problem you mentioned. |
This was referenced Feb 21, 2023
liangyepianzhou
pushed a commit
that referenced
this pull request
Feb 25, 2023
…t redevlier duplicate messages (#14327) If individual ack batch message with transaction and abort this transaction, we will redeliver this message. but this batch message some bit sit are acked by another transaction and re consume this bit sit will produce `TransactionConflictException`, we don't need to redeliver this bit sit witch is acked by another transaction. if batch have batch size 5 1. txn1 ack 0, 1 the ackSet is 00111 2. txn2 ack 2 3 4 the ack Set is 11000 3. abort txn2 redeliver this position is 00111 4. but now we don't filter txn1 ackSet so redeliver this position bitSet is 111111 When filter the message we should filter the bit sit witch is real ack or in pendingAck state add the test (cherry picked from commit e0c0d5e)
Member
|
@congbobo184 Can you help cherry-pick it to branch 2.9? |
nicoloboschi
pushed a commit
to datastax/pulsar
that referenced
this pull request
Feb 28, 2023
…t redevlier duplicate messages (apache#14327) If individual ack batch message with transaction and abort this transaction, we will redeliver this message. but this batch message some bit sit are acked by another transaction and re consume this bit sit will produce `TransactionConflictException`, we don't need to redeliver this bit sit witch is acked by another transaction. if batch have batch size 5 1. txn1 ack 0, 1 the ackSet is 00111 2. txn2 ack 2 3 4 the ack Set is 11000 3. abort txn2 redeliver this position is 00111 4. but now we don't filter txn1 ackSet so redeliver this position bitSet is 111111 When filter the message we should filter the bit sit witch is real ack or in pendingAck state add the test (cherry picked from commit e0c0d5e) (cherry picked from commit 90b4f86)
Technoboy-
pushed a commit
that referenced
this pull request
Mar 6, 2023
…t redevlier duplicate messages (#14327) ### Motivation If individual ack batch message with transaction and abort this transaction, we will redeliver this message. but this batch message some bit sit are acked by another transaction and re consume this bit sit will produce `TransactionConflictException`, we don't need to redeliver this bit sit witch is acked by another transaction. if batch have batch size 5 1. txn1 ack 0, 1 the ackSet is 00111 2. txn2 ack 2 3 4 the ack Set is 11000 3. abort txn2 redeliver this position is 00111 4. but now we don't filter txn1 ackSet so redeliver this position bitSet is 111111 ### Modifications When filter the message we should filter the bit sit witch is real ack or in pendingAck state ### Verifying this change add the test
Member
|
As discussed on the mailing list https://lists.apache.org/thread/w4jzk27qhtosgsz7l9bmhf1t7o9mxjhp, there is no plan to release 2.9.6, so I am going to remove the release/2.9.6 label |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
9 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
If individual ack batch message with transaction and abort this transaction, we will redeliver this message. but this batch message some bit sit are acked by another transaction and re consume this bit sit will produce
TransactionConflictException, we don't need to redeliver this bit sit witch is acked by another transaction.if batch have batch size 5
Modifications
When filter the message we should filter the bit sit witch is real ack or in pendingAck state
Verifying this change
add the test
Does this pull request potentially affect one of the following parts:
If
yeswas chosen, please highlight the changesDocumentation
Does this pull request introduce a new feature? (yes)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
If a feature is not applicable for documentation, explain why?
If a feature is not documented yet in this PR, please create a followup issue for adding the documentation
doc-not-needed