[Fix](Compaction) Cumulative Points Not Incrementing After Delete Operation #47282
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Collaborator
Author
|
run buildall |
TPC-H: Total hot run time: 32428 ms |
TPC-DS: Total hot run time: 194719 ms |
ClickBench: Total hot run time: 30.24 s |
|
TeamCity be ut coverage result: |
Contributor
|
PR approved by at least one committer and no changes requested. |
github-actions
bot
added
approved
Contributor
|
PR approved by anyone and no changes requested. |
Yukang-Lian
force-pushed
the
Fix-Delete-Compaction
branch
from
4710188 to
aea7baa
Compare
Collaborator
Author
|
run buildall |
github-actions
bot
removed
the
approved
TPC-H: Total hot run time: 34222 ms |
TPC-DS: Total hot run time: 186547 ms |
ClickBench: Total hot run time: 30.5 s |
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
Collaborator
Author
|
run buildall |
TPC-H: Total hot run time: 34271 ms |
TPC-DS: Total hot run time: 186187 ms |
ClickBench: Total hot run time: 31.29 s |
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
luwei16
approved these changes
Apr 1, 2025
gavinchou
approved these changes
Apr 1, 2025
Contributor
|
PR approved by at least one committer and no changes requested. |
github-actions
bot
added
the
approved
github-actions bot
pushed a commit
that referenced
this pull request
Apr 1, 2025
…ration (#47282) When a tablet's first rowset is a delete operation followed by data rowsets, the cumulative compaction process currently skips the delete rowset and only compacts the subsequent data rowsets. This leads to a failure in incrementing the cumulative point due to the condition output_rowset->start_version() == last_cumulative_point not being satisfied. In the current behavior, the cumulative point for the delete rowset only advances after all data rowsets have completed cumulative compaction. This creates an inefficient scenario where if continuous data ingestion occurs, all subsequent data rowsets are forced to participate in cumulative compaction, resulting in significant resource waste. This PR fixes the issue by properly handling the cumulative point advancement when encountering delete rowsets at the beginning of the version sequence.
Closed
koarz
pushed a commit
to koarz/doris
that referenced
this pull request
Jun 4, 2025
…ration (apache#47282) When a tablet's first rowset is a delete operation followed by data rowsets, the cumulative compaction process currently skips the delete rowset and only compacts the subsequent data rowsets. This leads to a failure in incrementing the cumulative point due to the condition output_rowset->start_version() == last_cumulative_point not being satisfied. In the current behavior, the cumulative point for the delete rowset only advances after all data rowsets have completed cumulative compaction. This creates an inefficient scenario where if continuous data ingestion occurs, all subsequent data rowsets are forced to participate in cumulative compaction, resulting in significant resource waste. This PR fixes the issue by properly handling the cumulative point advancement when encountering delete rowsets at the beginning of the version sequence.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
When a tablet's first rowset is a delete operation followed by data rowsets, the cumulative compaction process currently skips the delete rowset and only compacts the subsequent data rowsets. This leads to a failure in incrementing the cumulative point due to the condition output_rowset->start_version() == last_cumulative_point not being satisfied.
In the current behavior, the cumulative point for the delete rowset only advances after all data rowsets have completed cumulative compaction. This creates an inefficient scenario where if continuous data ingestion occurs, all subsequent data rowsets are forced to participate in cumulative compaction, resulting in significant resource waste.
This PR fixes the issue by properly handling the cumulative point advancement when encountering delete rowsets at the beginning of the version sequence.
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)