[Fix](merge-on-write) Correct the alignment process when the existing rows with same key has marked delete sign #24877
Conversation
|
run buildall |
|
clang-tidy review says "All clean, LGTM! 👍" |
|
TeamCity be ut coverage result: |
bobhan1
force-pushed
the
fix-partial-update-delete-sign
branch
from
aafaf08 to
354b933
Compare
|
run buildall |
|
clang-tidy review says "All clean, LGTM! 👍" |
|
TeamCity be ut coverage result: |
|
(From new machine)TeamCity pipeline, clickbench performance test result: |
bobhan1
force-pushed
the
fix-partial-update-delete-sign
branch
from
354b933 to
819937b
Compare
|
run buildall |
|
clang-tidy review says "All clean, LGTM! 👍" |
|
TeamCity be ut coverage result: |
|
(From new machine)TeamCity pipeline, clickbench performance test result: |
|
PR approved by at least one committer and no changes requested. |
github-actions
bot
added
approved
|
PR approved by anyone and no changes requested. |
xiaokang
added
merge_conflict
dev/2.0.3-merged
dev/2.0.3
and removed
dev/2.0.3
dev/2.0.3-merged
labels
…isting rows with same key has marked delete sign apache#24877"
… rows with same key has marked delete sign (apache#24877)
…n merge-on-write unique table (#25071) * pick "[Fix](Partial update) Fix wrong position using in segment writer #22782" * pick "[Enhancement](merge-on-write) use delete bitmap to mark delete for rows with delete sign when sequence column doesn't exist #24011" * pick "[Fix](merge-on-write) Correct the alignment process when the existing rows with same key has marked delete sign #24877"
…rows with delete sign (#36210) Issue Number: close #34296 1. When partial update filling in the missing fields, if a load job previously wrote data with a delete sign, it will also read out the data in the column with the delete sign, so that the newly written data will also become invisible 2. This problem was fixed in #24877, but was introduced again in #26721, and was never found because the case was changed to the wrong output in #26721. 3. The fix in #24877 didn't take into account the handling of concurrent conflicts in the publish phase, the current PR adds this part of the handling, and adds the corresponding case.
…rows with delete sign (#36210) Issue Number: close #34296 1. When partial update filling in the missing fields, if a load job previously wrote data with a delete sign, it will also read out the data in the column with the delete sign, so that the newly written data will also become invisible 2. This problem was fixed in #24877, but was introduced again in #26721, and was never found because the case was changed to the wrong output in #26721. 3. The fix in #24877 didn't take into account the handling of concurrent conflicts in the publish phase, the current PR adds this part of the handling, and adds the corresponding case.
…rows with delete sign (apache#36210) Issue Number: close apache#34296 1. When partial update filling in the missing fields, if a load job previously wrote data with a delete sign, it will also read out the data in the column with the delete sign, so that the newly written data will also become invisible 2. This problem was fixed in apache#24877, but was introduced again in apache#26721, and was never found because the case was changed to the wrong output in 3. The fix in apache#24877 didn't take into account the handling of concurrent conflicts in the publish phase, the current PR adds this part of the handling, and adds the corresponding case.
…rows with delete sign (apache#36210) Issue Number: close apache#34296 1. When partial update filling in the missing fields, if a load job previously wrote data with a delete sign, it will also read out the data in the column with the delete sign, so that the newly written data will also become invisible 2. This problem was fixed in apache#24877, but was introduced again in apache#26721, and was never found because the case was changed to the wrong output in 3. The fix in apache#24877 didn't take into account the handling of concurrent conflicts in the publish phase, the current PR adds this part of the handling, and adds the corresponding case.
5 participants
Proposed changes
When we insert a row with delete sign marked, it will not mark itself on the delete bitmap if the table has sequence column(see #24011). So if we delete a row using delete sign and then insert a row with same keys using partial update, it will wrongly fill the values in in the unmentioned columns in which the delete sign column will be filled with 1 and the row that should have been inserted will be deleted. So in this occasion we must check the delete sign column to check that if a row really exists in the table.
Further comments
If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...