Skip to content

Conversation

@freemandealer
Copy link
Contributor

@freemandealer freemandealer commented Feb 14, 2023

This patchset applies the following changes:

  • using vertical compaction machanism to do segcompaction
  • basic (WIP) refraction to separate segcompaction logic from BetaRowsetWriter
  • add segcompaction specific ut and regression tests

Proposed changes

Issue Number: close #xxx

Problem summary

Describe your changes.

Checklist(Required)

  • Does it affect the original behavior
  • Has unit tests been added
  • Has document been added or modified
  • Does it need to update dependencies
  • Is this PR support rollback (If NO, please explain WHY)

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

@freemandealer freemandealer force-pushed the segcompaction_feat_vertical branch from 838df6b to 1dfa99b Compare February 14, 2023 12:39
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

@freemandealer freemandealer force-pushed the segcompaction_feat_vertical branch from 1dfa99b to 9ceb4f7 Compare February 21, 2023 08:21
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

@freemandealer freemandealer force-pushed the segcompaction_feat_vertical branch from 9ceb4f7 to c2727c1 Compare February 21, 2023 11:21
@freemandealer freemandealer changed the title [WIP] segcompaction feat. verticalcompaction segcompaction featuring verticalcompaction Feb 21, 2023
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

@freemandealer freemandealer force-pushed the segcompaction_feat_vertical branch from 953aa2f to 42d150b Compare February 21, 2023 12:01
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

@freemandealer freemandealer force-pushed the segcompaction_feat_vertical branch from d711363 to 658b08b Compare February 24, 2023 03:28
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

@freemandealer freemandealer force-pushed the segcompaction_feat_vertical branch 2 times, most recently from 368c8e5 to 5517038 Compare February 24, 2023 08:19
@freemandealer
Copy link
Contributor Author

run buildall

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

@hello-stephen
Copy link
Contributor

hello-stephen commented Feb 24, 2023

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 34.11 seconds
stream load tsv: 468 seconds loaded 74807831229 Bytes, about 152 MB/s
stream load json: 38 seconds loaded 2358488459 Bytes, about 59 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 27 seconds loaded 861443392 Bytes, about 30 MB/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230228135527_clickbench_pr_105771.html

@freemandealer freemandealer force-pushed the segcompaction_feat_vertical branch from 5517038 to 5e2bbaf Compare February 27, 2023 11:47
zhannngchen
zhannngchen previously approved these changes Feb 28, 2023
Copy link
Contributor

@zhannngchen zhannngchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Signed-off-by: freemandealer <freeman.zhang1992@gmail.com>
@freemandealer freemandealer requested review from zhannngchen and removed request for yixiutt March 1, 2023 02:49
Copy link
Contributor

@zhannngchen zhannngchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 1, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Mar 1, 2023

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Mar 1, 2023

PR approved by anyone and no changes requested.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. area/vectorization dev/1.2.3 kind/test reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants