Skip to content

[Feature](Row Store) support partial columns row store in cloud mode #36658

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
eldenmoon merged 1 commit into from
Jun 25, 2024
Merged

[Feature](Row Store) support partial columns row store in cloud mode #36658

eldenmoon merged 1 commit into from
Jun 25, 2024

Conversation

@eldenmoon
Copy link
Member

@eldenmoon eldenmoon commented Jun 21, 2024 •
edited
Loading

previous PR:
#34089
Issue Number: close #xxx

@doris-robot
Copy link

doris-robot commented Jun 21, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@eldenmoon
Copy link
Member Author

eldenmoon commented Jun 21, 2024

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Jun 21, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

doris-robot commented Jun 21, 2024

TPC-H: Total hot run time: 39808 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7126c8e32f4425e9aa6edb011f33d6fb12eab7c3, data reload: false

------ Round 1 ----------------------------------
q1	17959	4440	4438	4438
q2	2574	202	208	202
q3	12194	1116	1100	1100
q4	10361	848	739	739
q5	8128	2680	2699	2680
q6	225	133	131	131
q7	944	610	596	596
q8	9215	2081	2043	2043
q9	8854	6475	6459	6459
q10	8960	3784	3700	3700
q11	443	256	236	236
q12	399	242	232	232
q13	17767	2999	2999	2999
q14	264	217	213	213
q15	534	483	475	475
q16	533	378	383	378
q17	961	710	642	642
q18	7953	7440	7436	7436
q19	7909	1283	1512	1283
q20	670	317	337	317
q21	4878	3194	3176	3176
q22	386	333	335	333
Total cold run time: 122111 ms
Total hot run time: 39808 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4444	4312	4273	4273
q2	371	262	270	262
q3	2984	2724	2738	2724
q4	1866	1639	1607	1607
q5	5282	5305	5248	5248
q6	220	124	127	124
q7	2125	1685	1743	1685
q8	3163	3318	3383	3318
q9	8317	8395	8350	8350
q10	3900	3729	3650	3650
q11	582	487	475	475
q12	783	619	591	591
q13	17648	2977	2991	2977
q14	278	251	267	251
q15	510	476	476	476
q16	478	417	416	416
q17	1747	1495	1474	1474
q18	7641	7572	7313	7313
q19	1688	1610	1564	1564
q20	2001	1796	1798	1796
q21	4901	4769	4763	4763
q22	629	559	566	559
Total cold run time: 71558 ms
Total hot run time: 53896 ms

@doris-robot
Copy link

doris-robot commented Jun 21, 2024

TeamCity be ut coverage result:
Function Coverage: 36.47% (9007/24699)
Line Coverage: 28.01% (73864/263737)
Region Coverage: 27.48% (38357/139589)
Branch Coverage: 24.18% (19552/80858)
Coverage Report: http://coverage.selectdb-in.cc/coverage/7126c8e32f4425e9aa6edb011f33d6fb12eab7c3_7126c8e32f4425e9aa6edb011f33d6fb12eab7c3/report/index.html

@doris-robot
Copy link

doris-robot commented Jun 21, 2024

TPC-DS: Total hot run time: 172407 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7126c8e32f4425e9aa6edb011f33d6fb12eab7c3, data reload: false

query1	920	375	375	375
query2	6469	2494	2257	2257
query3	6644	212	217	212
query4	18804	17330	17411	17330
query5	4181	497	505	497
query6	269	163	172	163
query7	4600	309	296	296
query8	319	292	300	292
query9	8530	2447	2414	2414
query10	605	294	285	285
query11	10397	10058	10185	10058
query12	133	86	85	85
query13	1628	358	356	356
query14	9968	6608	7557	6608
query15	245	200	192	192
query16	7980	264	261	261
query17	1904	542	509	509
query18	1922	268	265	265
query19	192	161	161	161
query20	94	86	84	84
query21	213	154	125	125
query22	4292	4140	3991	3991
query23	33692	33231	32965	32965
query24	11106	2807	2736	2736
query25	610	356	361	356
query26	1151	151	148	148
query27	2996	321	320	320
query28	7553	2066	2066	2066
query29	893	625	629	625
query30	282	157	155	155
query31	960	724	756	724
query32	91	55	56	55
query33	774	284	284	284
query34	954	477	460	460
query35	750	614	616	614
query36	1073	934	975	934
query37	174	72	75	72
query38	2920	2707	2750	2707
query39	840	783	785	783
query40	210	129	125	125
query41	55	54	56	54
query42	121	99	105	99
query43	595	559	561	559
query44	1175	731	741	731
query45	194	171	163	163
query46	1076	715	721	715
query47	1844	1773	1781	1773
query48	375	299	294	294
query49	1139	417	422	417
query50	760	381	391	381
query51	6941	6860	6788	6788
query52	104	100	87	87
query53	376	291	296	291
query54	863	440	449	440
query55	75	73	75	73
query56	272	259	267	259
query57	1131	1029	1047	1029
query58	240	236	246	236
query59	3512	3417	3232	3232
query60	308	294	277	277
query61	94	89	93	89
query62	641	437	456	437
query63	321	296	290	290
query64	8972	2262	1740	1740
query65	3145	3325	3117	3117
query66	953	328	348	328
query67	15601	15322	14926	14926
query68	4553	523	528	523
query69	464	292	306	292
query70	1160	1131	1089	1089
query71	395	265	272	265
query72	7142	5331	5905	5331
query73	741	324	322	322
query74	5941	5562	5528	5528
query75	3404	2681	2632	2632
query76	2365	999	896	896
query77	484	324	307	307
query78	10368	9841	9648	9648
query79	2785	514	519	514
query80	2007	497	484	484
query81	582	217	227	217
query82	869	110	104	104
query83	337	179	182	179
query84	275	90	89	89
query85	1695	358	326	326
query86	484	337	290	290
query87	3278	3101	3078	3078
query88	3869	2348	2348	2348
query89	466	389	393	389
query90	1792	193	194	193
query91	145	110	113	110
query92	60	50	51	50
query93	2611	515	503	503
query94	1194	201	204	201
query95	417	331	342	331
query96	604	267	268	267
query97	3243	3070	3084	3070
query98	226	337	203	203
query99	1381	858	890	858
Total cold run time: 273558 ms
Total hot run time: 172407 ms

@doris-robot
Copy link

doris-robot commented Jun 21, 2024

ClickBench: Total hot run time: 31.24 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7126c8e32f4425e9aa6edb011f33d6fb12eab7c3, data reload: false

query1	0.05	0.04	0.03
query2	0.09	0.04	0.04
query3	0.23	0.05	0.05
query4	1.68	0.07	0.07
query5	0.49	0.48	0.50
query6	1.13	0.74	0.74
query7	0.02	0.02	0.02
query8	0.06	0.04	0.04
query9	0.55	0.50	0.49
query10	0.56	0.54	0.53
query11	0.15	0.11	0.12
query12	0.15	0.13	0.13
query13	0.59	0.59	0.59
query14	0.80	0.78	0.79
query15	0.86	0.82	0.82
query16	0.37	0.35	0.38
query17	1.03	1.02	1.04
query18	0.21	0.27	0.23
query19	1.91	1.71	1.83
query20	0.01	0.01	0.01
query21	15.42	0.67	0.66
query22	3.87	7.15	2.46
query23	18.31	1.40	1.23
query24	2.10	0.25	0.24
query25	0.15	0.09	0.08
query26	0.27	0.18	0.19
query27	0.09	0.08	0.08
query28	13.22	1.03	1.00
query29	12.61	3.29	3.25
query30	0.30	0.08	0.08
query31	2.83	0.40	0.39
query32	3.26	0.48	0.48
query33	2.87	2.92	2.89
query34	17.20	4.41	4.44
query35	4.48	4.51	4.48
query36	0.65	0.46	0.46
query37	0.19	0.16	0.16
query38	0.16	0.15	0.15
query39	0.05	0.04	0.04
query40	0.17	0.14	0.18
query41	0.10	0.05	0.06
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.35 s
Total hot run time: 31.24 s

dataroaring
dataroaring approved these changes Jun 24, 2024
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 24, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Jun 24, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Jun 24, 2024

PR approved by anyone and no changes requested.

zhannngchen
zhannngchen requested changes Jun 24, 2024
View reviewed changes
Copy link
Contributor

@zhannngchen zhannngchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no regresion case?

@eldenmoon
Copy link
Member Author

eldenmoon commented Jun 24, 2024

@zhannngchen already cases in point_query_p0/test_rowstore.groovy

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jun 24, 2024
@eldenmoon eldenmoon requested a review from zhannngchen June 24, 2024 14:15
zhannngchen
zhannngchen approved these changes Jun 25, 2024
View reviewed changes
Copy link
Contributor

@zhannngchen zhannngchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 25, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Jun 25, 2024

PR approved by at least one committer and no changes requested.

@eldenmoon eldenmoon merged commit 6551bf6 into apache:master Jun 25, 2024
dataroaring pushed a commit that referenced this pull request Jun 26, 2024
@eldenmoon @dataroaring
[Feature](Row Store) support partial columns row store in cloud mode (#…
3dc13c8 
…36658)

previous PR:
#34089
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zhannngchen zhannngchen zhannngchen approved these changes

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.0-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@eldenmoon @doris-robot @zhannngchen @dataroaring