[cloud](Variant) limit column size in MetaService and support handle rowset schema in backend side #41482

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merged

xiaokang merged 2 commits into apache:master from eldenmoon:cloud-lm-ar

Nov 1, 2024

Member

eldenmoon commented Sep 29, 2024 •

edited

Loading

in local mode we limit column size in commit_txn phase, in cloud mode we limit it in write_schema_dict which in commit_rowset phase
add config to decide whether fill schema in BE side or MS side

doris-robot commented Sep 29, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

Member Author

eldenmoon commented Sep 29, 2024

run buildall

eldenmoon force-pushed the cloud-lm-ar branch from 4b900cf to 262f7eb Compare

September 29, 2024 17:53

Member Author

eldenmoon commented Sep 29, 2024

run buildall

wm1581066 requested a review from gavinchou

September 30, 2024 02:07

wm1581066 added the usercase label

gavinchou reviewed

View reviewed changes

cloud/src/meta-service/meta_service_schema.cpp Show resolved Hide resolved

gavinchou requested changes

View reviewed changes

cloud/src/common/config.h Outdated Show resolved Hide resolved


          [cloud](Variant) limit column size in MetaService

acda4df

in local mode we limit column size in commit_txn phase, in cloud mode we limit it in `write_schema_dict` which in commit_rowset phase

eldenmoon force-pushed the cloud-lm-ar branch 2 times, most recently from b1f8c04 to acda4df Compare

October 11, 2024 07:11

Contributor

github-actions bot commented Oct 11, 2024

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment

Contributor

github-actions bot commented Oct 11, 2024

clang-tidy review says "All clean, LGTM! 👍"

Member Author

eldenmoon commented Oct 21, 2024

run buildall

Contributor

github-actions bot commented Oct 21, 2024

clang-tidy review says "All clean, LGTM! 👍"

eldenmoon force-pushed the cloud-lm-ar branch from 366aa9d to c26c607 Compare

October 21, 2024 08:05

Contributor

github-actions bot commented Oct 21, 2024

clang-tidy review says "All clean, LGTM! 👍"

doris-robot commented Oct 21, 2024

TeamCity be ut coverage result:
Function Coverage: 37.48% (9708/25903)
Line Coverage: 28.76% (80650/280471)
Region Coverage: 28.19% (41730/148016)
Branch Coverage: 24.77% (21214/85650)
Coverage Report: http://coverage.selectdb-in.cc/coverage/366aa9d61e917e19e0bc1d478018ec51553aaa01_366aa9d61e917e19e0bc1d478018ec51553aaa01/report/index.html


          optimize get_rowset fill dict columns with variant

9ee812c

eldenmoon force-pushed the cloud-lm-ar branch from c26c607 to 9ee812c Compare

October 22, 2024 13:22

Member Author

eldenmoon commented Oct 22, 2024

run buildall

Contributor

github-actions bot commented Oct 22, 2024

clang-tidy review says "All clean, LGTM! 👍"

eldenmoon requested a review from gavinchou

October 22, 2024 13:35

doris-robot commented Oct 22, 2024

TeamCity be ut coverage result:
Function Coverage: 37.47% (9708/25911)
Line Coverage: 28.73% (80534/280308)
Region Coverage: 28.15% (41638/147916)
Branch Coverage: 24.72% (21156/85578)
Coverage Report: http://coverage.selectdb-in.cc/coverage/9ee812c2b11891b938ea73e548b998cafd85d9e8_9ee812c2b11891b938ea73e548b998cafd85d9e8/report/index.html

eldenmoon changed the title ~~[cloud](Variant) limit column size in MetaService~~ [cloud](Variant) limit column size in MetaService and support handle rowset schema in backend side

eldenmoon added the dev/3.0.x label

xiaokang reviewed

View reviewed changes

be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp Show resolved Hide resolved

be/src/cloud/pb_convert.cpp Show resolved Hide resolved

cloud/src/common/config.h Show resolved Hide resolved

cloud/src/meta-service/meta_service_schema.cpp Show resolved Hide resolved

gavinchou approved these changes

View reviewed changes

Contributor

gavinchou commented Oct 31, 2024

LGTM

github-actions bot added the approved label

Contributor

github-actions bot commented Oct 31, 2024

PR approved by at least one committer and no changes requested.

github-actions bot added the reviewed label

Contributor

github-actions bot commented Oct 31, 2024

PR approved by anyone and no changes requested.

eldenmoon added dev/3.0.x and removed dev/3.0.x labels

xiaokang approved these changes

View reviewed changes

Contributor

xiaokang left a comment

LGTM

xiaokang merged commit 2092b44 into apache:master

github-actions bot pushed a commit that referenced this pull request


          [cloud](Variant) limit column size in MetaService and support handle …

f85ade1

…rowset schema in backend side (#41482)

1. in local mode we limit column size in commit_txn phase, in cloud mode
we limit it in `write_schema_dict` which in commit_rowset phase
2. add config to decide whether fill schema in BE side or MS side

github-actions bot mentioned this pull request

[cloud](Variant) limit column size in MetaService and support handle rowset schema in backend side(#41482) #43093

Merged

eldenmoon deleted the cloud-lm-ar branch

November 4, 2024 02:11

eldenmoon pushed a commit that referenced this pull request


          [cloud](Variant) limit column size in MetaService and support handle …

d9f270a

…rowset schema in backend side(#41482) (#43093)

PR Body: 1. in local mode we limit column size in commit_txn phase, in
cloud mode we limit it in `write_schema_dict` which in commit_rowset
phase
2. add config to decide whether fill schema in BE side or MS side

 
 Cherry-picked from #41482

eldenmoon added dev/3.0.3-merged and removed dev/3.0.x labels

gavinchou added the cir label

gavinchou mentioned this pull request

Release Note 3.0.3 #44522

Closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved cir dev/3.0.3-merged reviewed usercase