Skip to content

[Improvement](materialized-view) support multiple agg function have same base table slot #33774

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
BiteTheDDDDt merged 5 commits into from
Apr 18, 2024
Merged

[Improvement](materialized-view) support multiple agg function have same base table slot #33774

BiteTheDDDDt merged 5 commits into from
Apr 18, 2024

Conversation

@BiteTheDDDDt
Copy link
Contributor

@BiteTheDDDDt BiteTheDDDDt commented Apr 17, 2024

Proposed changes

support multiple agg function have same base table slot

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

doris-robot commented Apr 17, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@BiteTheDDDDt
Copy link
Contributor Author

BiteTheDDDDt commented Apr 17, 2024

run buildall

@BiteTheDDDDt
Copy link
Contributor Author

BiteTheDDDDt commented Apr 17, 2024

run buildall

@morrySnow morrySnow requested a review from starocean999 April 17, 2024 09:16
@BiteTheDDDDt
Copy link
Contributor Author

BiteTheDDDDt commented Apr 17, 2024

run buildall

@BiteTheDDDDt
Copy link
Contributor Author

BiteTheDDDDt commented Apr 17, 2024

run buildall

@BiteTheDDDDt
Copy link
Contributor Author

BiteTheDDDDt commented Apr 17, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Apr 17, 2024

TPC-H: Total hot run time: 39082 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit baf914c954b8d634d436f30fd9e387e33c453e3a, data reload: false

------ Round 1 ----------------------------------
q1	17723	4485	4343	4343
q2	2620	196	185	185
q3	10979	1196	1172	1172
q4	10508	857	729	729
q5	7847	2749	2679	2679
q6	226	140	141	140
q7	1030	612	613	612
q8	9348	2056	2046	2046
q9	8011	7146	7048	7048
q10	8580	3531	3526	3526
q11	435	235	230	230
q12	492	226	213	213
q13	17763	2965	2996	2965
q14	260	220	224	220
q15	514	486	458	458
q16	519	378	378	378
q17	967	623	725	623
q18	7411	6852	6736	6736
q19	3449	1525	1480	1480
q20	639	309	309	309
q21	3571	2679	2796	2679
q22	369	315	311	311
Total cold run time: 113261 ms
Total hot run time: 39082 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4326	4278	4223	4223
q2	373	264	268	264
q3	2956	2725	2699	2699
q4	1893	1569	1573	1569
q5	5362	5313	5281	5281
q6	207	123	125	123
q7	2223	1907	1883	1883
q8	3206	3326	3336	3326
q9	9251	9219	9217	9217
q10	3897	3675	3726	3675
q11	589	469	482	469
q12	751	582	574	574
q13	16296	2958	2964	2958
q14	298	270	276	270
q15	506	467	473	467
q16	469	420	432	420
q17	1780	1474	1447	1447
q18	7589	7418	7368	7368
q19	1631	1503	1537	1503
q20	1952	1775	1746	1746
q21	9936	4822	4763	4763
q22	529	453	460	453
Total cold run time: 76020 ms
Total hot run time: 54698 ms

@doris-robot
Copy link

doris-robot commented Apr 17, 2024

TPC-DS: Total hot run time: 184115 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit baf914c954b8d634d436f30fd9e387e33c453e3a, data reload: false

query1	909	373	367	367
query2	6494	2533	2509	2509
query3	6649	208	207	207
query4	24513	21436	21275	21275
query5	4176	403	410	403
query6	269	167	173	167
query7	4584	286	287	286
query8	222	174	177	174
query9	8475	2328	2304	2304
query10	572	243	243	243
query11	14769	14168	14135	14135
query12	132	90	83	83
query13	1664	361	376	361
query14	9486	7417	7490	7417
query15	227	182	189	182
query16	8007	267	288	267
query17	1903	591	592	591
query18	2031	279	270	270
query19	210	145	154	145
query20	90	84	89	84
query21	194	126	123	123
query22	5025	4893	4844	4844
query23	33690	33131	33158	33131
query24	11797	2964	2906	2906
query25	661	362	376	362
query26	1737	157	146	146
query27	3028	305	300	300
query28	7589	2006	1994	1994
query29	1037	585	599	585
query30	280	173	166	166
query31	974	726	720	720
query32	97	55	51	51
query33	731	272	243	243
query34	1014	471	467	467
query35	837	677	679	677
query36	1056	933	911	911
query37	191	77	70	70
query38	3374	3196	3175	3175
query39	1577	1552	1537	1537
query40	284	126	124	124
query41	47	45	43	43
query42	104	94	96	94
query43	584	535	552	535
query44	1211	718	714	714
query45	285	238	256	238
query46	1063	717	716	716
query47	1896	1866	1843	1843
query48	376	291	293	291
query49	1158	370	369	369
query50	741	372	368	368
query51	6681	6571	6669	6571
query52	101	87	88	87
query53	352	283	279	279
query54	310	233	219	219
query55	75	71	72	71
query56	238	251	219	219
query57	1240	1148	1140	1140
query58	218	191	192	191
query59	3227	3091	3150	3091
query60	252	234	228	228
query61	89	88	88	88
query62	661	439	436	436
query63	305	274	276	274
query64	6363	3789	3820	3789
query65	3112	3032	3049	3032
query66	1383	350	341	341
query67	15276	14973	14868	14868
query68	5212	531	527	527
query69	482	298	298	298
query70	1261	1161	1149	1149
query71	1396	1265	1264	1264
query72	6564	2642	2413	2413
query73	711	316	316	316
query74	6820	6493	6428	6428
query75	3367	2675	2628	2628
query76	3376	912	972	912
query77	426	262	263	262
query78	11013	10208	10205	10205
query79	2665	523	514	514
query80	2007	426	422	422
query81	537	246	242	242
query82	765	88	94	88
query83	323	172	168	168
query84	269	87	83	83
query85	1966	271	311	271
query86	481	290	287	287
query87	3504	3270	3280	3270
query88	4250	2333	2396	2333
query89	462	366	378	366
query90	1996	179	184	179
query91	124	94	129	94
query92	62	46	46	46
query93	3803	513	489	489
query94	1261	177	178	177
query95	395	298	296	296
query96	571	271	266	266
query97	3196	2905	2946	2905
query98	231	212	221	212
query99	1217	868	885	868
Total cold run time: 288616 ms
Total hot run time: 184115 ms

@doris-robot
Copy link

doris-robot commented Apr 17, 2024

ClickBench: Total hot run time: 29.85 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit baf914c954b8d634d436f30fd9e387e33c453e3a, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.04
query4	1.67	0.06	0.07
query5	0.48	0.49	0.49
query6	1.45	0.72	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.05
query9	0.55	0.48	0.49
query10	0.53	0.56	0.55
query11	0.15	0.12	0.11
query12	0.14	0.12	0.12
query13	0.61	0.58	0.58
query14	0.78	0.76	0.77
query15	0.81	0.81	0.80
query16	0.37	0.35	0.35
query17	0.93	0.96	0.98
query18	0.23	0.24	0.23
query19	1.75	1.64	1.69
query20	0.02	0.01	0.01
query21	15.43	0.64	0.64
query22	4.63	7.75	1.57
query23	18.25	1.46	1.29
query24	1.38	0.35	0.29
query25	0.16	0.08	0.09
query26	0.27	0.16	0.17
query27	0.08	0.07	0.07
query28	13.33	1.01	0.97
query29	12.55	3.23	3.22
query30	0.26	0.06	0.06
query31	2.85	0.38	0.37
query32	3.30	0.46	0.45
query33	2.77	2.79	2.81
query34	17.21	4.44	4.36
query35	4.47	4.48	4.51
query36	0.66	0.45	0.48
query37	0.20	0.15	0.15
query38	0.14	0.14	0.14
query39	0.05	0.04	0.04
query40	0.18	0.14	0.14
query41	0.10	0.04	0.04
query42	0.05	0.04	0.04
query43	0.04	0.03	0.04
Total cold run time: 109.25 s
Total hot run time: 29.85 s

@doris-robot
Copy link

doris-robot commented Apr 17, 2024

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit baf914c954b8d634d436f30fd9e387e33c453e3a with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      33 seconds loaded 861443392 Bytes, about 24 MB/s
Insert into select:       13.5 seconds inserted 10000000 Rows, about 740K ops/s

@github-actions
Copy link
Contributor

github-actions bot commented Apr 18, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 18, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Apr 18, 2024

PR approved by anyone and no changes requested.

@BiteTheDDDDt BiteTheDDDDt merged commit ce2c1ad into apache:master Apr 18, 2024
yiguolei pushed a commit that referenced this pull request Apr 18, 2024
@BiteTheDDDDt @Doris-Extras
[Improvement](materialized-view) support multiple agg function have s…
8c535c5 
…ame base table slot (#33774)

support multiple agg function have same base table slot
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@starocean999 starocean999 starocean999 approved these changes

@zhangstar333 zhangstar333 zhangstar333 approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@BiteTheDDDDt @doris-robot @starocean999 @zhangstar333