Skip to content

Conversation

@zzzxl1993
Copy link
Contributor

@Thearas
Copy link
Contributor

Thearas commented Jan 7, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@zzzxl1993
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41139 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d548e4b2f3281d5d0f0975c8e4e52e16dcea8304, data reload: false

------ Round 1 ----------------------------------
q1	17672	7561	7437	7437
q2	2492	181	198	181
q3	11112	1117	1135	1117
q4	10219	725	753	725
q5	7748	2892	2777	2777
q6	239	146	146	146
q7	972	610	614	610
q8	9364	1988	1960	1960
q9	6717	6410	6434	6410
q10	7010	2271	2347	2271
q11	462	266	271	266
q12	410	219	208	208
q13	17807	3021	3035	3021
q14	236	210	207	207
q15	583	538	545	538
q16	663	604	602	602
q17	976	595	575	575
q18	7255	6848	6596	6596
q19	1408	1049	1042	1042
q20	485	200	195	195
q21	4044	3272	3266	3266
q22	1113	1014	989	989
Total cold run time: 108987 ms
Total hot run time: 41139 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7294	7194	7193	7193
q2	328	231	231	231
q3	2978	3017	3000	3000
q4	2057	1819	1841	1819
q5	5735	5741	5709	5709
q6	221	145	138	138
q7	2183	1787	1871	1787
q8	3430	3502	3502	3502
q9	8850	8804	8865	8804
q10	3576	3554	3553	3553
q11	607	494	489	489
q12	821	601	591	591
q13	14292	3157	3136	3136
q14	291	271	266	266
q15	571	528	524	524
q16	676	648	662	648
q17	1886	1605	1609	1605
q18	8361	7790	7489	7489
q19	1677	1521	1496	1496
q20	2061	1816	1802	1802
q21	5274	5159	5114	5114
q22	1081	998	1009	998
Total cold run time: 74250 ms
Total hot run time: 59894 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191910 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d548e4b2f3281d5d0f0975c8e4e52e16dcea8304, data reload: false

query1	979	392	364	364
query2	6508	2120	2024	2024
query3	6704	221	223	221
query4	33889	23620	23511	23511
query5	4393	466	453	453
query6	276	175	180	175
query7	4630	311	303	303
query8	279	226	233	226
query9	9421	2697	2701	2697
query10	502	252	261	252
query11	18168	15098	15291	15098
query12	155	103	104	103
query13	1643	435	420	420
query14	9750	7357	7022	7022
query15	251	175	183	175
query16	8051	435	489	435
query17	1625	585	564	564
query18	2120	317	326	317
query19	233	170	164	164
query20	118	110	110	110
query21	209	106	104	104
query22	4594	4208	4196	4196
query23	34583	34130	34224	34130
query24	12048	2790	2875	2790
query25	700	382	408	382
query26	1847	166	173	166
query27	2895	346	351	346
query28	7938	2476	2433	2433
query29	1052	435	413	413
query30	333	166	163	163
query31	1016	801	811	801
query32	92	59	57	57
query33	787	285	290	285
query34	955	499	509	499
query35	849	719	725	719
query36	1086	950	926	926
query37	175	77	78	77
query38	3982	3883	3858	3858
query39	1486	1422	1543	1422
query40	284	97	97	97
query41	52	49	49	49
query42	113	104	98	98
query43	534	478	484	478
query44	1297	792	786	786
query45	181	171	168	168
query46	1138	697	703	697
query47	1905	1834	1830	1830
query48	469	376	388	376
query49	1261	392	389	389
query50	821	407	405	405
query51	7203	7242	7227	7227
query52	100	88	91	88
query53	259	193	194	193
query54	1290	466	470	466
query55	78	79	81	79
query56	268	255	252	252
query57	1241	1105	1113	1105
query58	242	208	216	208
query59	3338	2918	3213	2918
query60	292	253	256	253
query61	114	112	112	112
query62	878	664	660	660
query63	231	190	187	187
query64	5210	668	621	621
query65	3299	3215	3191	3191
query66	1440	316	309	309
query67	16031	15887	15616	15616
query68	3215	597	578	578
query69	400	279	270	270
query70	1182	1132	1130	1130
query71	331	251	256	251
query72	6129	4041	4050	4041
query73	769	358	351	351
query74	9851	8957	8888	8888
query75	3359	2611	2683	2611
query76	1983	1106	1005	1005
query77	431	291	277	277
query78	10327	9771	9514	9514
query79	1104	601	594	594
query80	656	471	446	446
query81	521	250	242	242
query82	211	120	119	119
query83	247	157	156	156
query84	248	84	80	80
query85	1050	368	353	353
query86	325	306	281	281
query87	4475	4304	4327	4304
query88	3645	2413	2379	2379
query89	393	290	304	290
query90	2066	189	192	189
query91	195	166	161	161
query92	67	54	56	54
query93	1039	555	548	548
query94	818	304	303	303
query95	377	274	268	268
query96	618	281	288	281
query97	3298	3204	3193	3193
query98	215	205	212	205
query99	1499	1311	1289	1289
Total cold run time: 296935 ms
Total hot run time: 191910 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.87 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d548e4b2f3281d5d0f0975c8e4e52e16dcea8304, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.02	0.04
query3	0.24	0.06	0.06
query4	1.63	0.10	0.10
query5	0.54	0.53	0.54
query6	1.13	0.73	0.73
query7	0.03	0.02	0.01
query8	0.04	0.03	0.04
query9	0.56	0.51	0.50
query10	0.55	0.55	0.57
query11	0.14	0.10	0.10
query12	0.15	0.14	0.11
query13	0.62	0.61	0.60
query14	2.92	2.94	2.98
query15	0.91	0.83	0.82
query16	0.39	0.38	0.37
query17	1.01	1.00	1.05
query18	0.24	0.22	0.21
query19	1.96	1.87	2.00
query20	0.01	0.02	0.01
query21	15.37	0.58	0.57
query22	2.36	2.13	1.81
query23	17.17	0.98	0.84
query24	2.93	0.89	1.19
query25	0.27	0.10	0.05
query26	0.52	0.13	0.13
query27	0.05	0.04	0.04
query28	10.59	1.10	1.09
query29	12.59	3.29	3.25
query30	0.25	0.07	0.06
query31	2.85	0.39	0.38
query32	3.25	0.45	0.46
query33	2.98	2.97	3.05
query34	16.89	4.47	4.58
query35	4.51	4.48	4.59
query36	0.68	0.49	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.03	0.02
query40	0.16	0.13	0.13
query41	0.08	0.02	0.03
query42	0.04	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.93 s
Total hot run time: 32.87 s

@zzzxl1993
Copy link
Contributor Author

run buildall

@zzzxl1993 zzzxl1993 force-pushed the branch-3.0.202401022034 branch from d548e4b to bb5cf73 Compare January 16, 2025 03:18
@zzzxl1993
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40774 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bb5cf73daf1ac8ca1f594a1c8436da2b3f07ace4, data reload: false

------ Round 1 ----------------------------------
q1	17585	7393	7499	7393
q2	2060	181	171	171
q3	10717	1097	1168	1097
q4	10550	732	722	722
q5	7773	2821	2883	2821
q6	241	146	146	146
q7	966	631	619	619
q8	9361	1943	2018	1943
q9	6570	6352	6418	6352
q10	7040	2305	2311	2305
q11	471	267	269	267
q12	399	217	213	213
q13	17819	2975	3013	2975
q14	253	218	211	211
q15	555	513	507	507
q16	680	633	626	626
q17	971	551	529	529
q18	7122	6659	6770	6659
q19	1395	1120	979	979
q20	465	207	197	197
q21	3976	3214	3069	3069
q22	1114	984	973	973
Total cold run time: 108083 ms
Total hot run time: 40774 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7217	7218	7308	7218
q2	331	239	238	238
q3	2898	2910	2971	2910
q4	2085	1807	1894	1807
q5	5730	5675	5749	5675
q6	223	141	144	141
q7	2278	1815	1848	1815
q8	3364	3565	3558	3558
q9	8832	8900	8859	8859
q10	3574	3514	3543	3514
q11	597	529	493	493
q12	803	613	601	601
q13	8808	3166	3152	3152
q14	326	267	270	267
q15	579	526	527	526
q16	712	688	669	669
q17	1836	1614	1605	1605
q18	8335	7705	7660	7660
q19	1664	1583	1524	1524
q20	2117	1907	1899	1899
q21	5586	5250	5413	5250
q22	1116	1008	1018	1008
Total cold run time: 69011 ms
Total hot run time: 60389 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196914 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bb5cf73daf1ac8ca1f594a1c8436da2b3f07ace4, data reload: false

query1	1336	923	897	897
query2	6258	2095	2073	2073
query3	10831	4315	4202	4202
query4	65911	28412	23435	23435
query5	5056	457	458	457
query6	416	189	182	182
query7	5515	314	324	314
query8	317	232	226	226
query9	8603	2689	2690	2689
query10	449	282	263	263
query11	17258	15156	15812	15156
query12	166	108	104	104
query13	1462	471	431	431
query14	10820	7039	6769	6769
query15	208	185	196	185
query16	7097	491	522	491
query17	1091	574	598	574
query18	1824	337	332	332
query19	209	163	162	162
query20	119	111	108	108
query21	213	104	106	104
query22	4682	4558	4351	4351
query23	35039	34723	34338	34338
query24	6173	2977	2985	2977
query25	556	426	417	417
query26	661	185	175	175
query27	1944	373	367	367
query28	4099	2527	2471	2471
query29	694	451	449	449
query30	247	164	170	164
query31	1005	811	831	811
query32	65	63	56	56
query33	434	294	295	294
query34	917	522	550	522
query35	843	723	734	723
query36	1094	980	963	963
query37	119	71	74	71
query38	4118	4002	4069	4002
query39	1554	1529	1458	1458
query40	197	102	105	102
query41	55	52	51	51
query42	116	107	104	104
query43	530	498	499	498
query44	1177	826	840	826
query45	193	171	173	171
query46	1131	730	742	730
query47	2028	1851	1915	1851
query48	475	388	386	386
query49	752	402	407	402
query50	863	434	437	434
query51	7353	7157	7189	7157
query52	99	87	90	87
query53	257	182	186	182
query54	550	456	463	456
query55	76	79	74	74
query56	267	239	253	239
query57	1229	1107	1127	1107
query58	221	201	210	201
query59	3143	3010	2823	2823
query60	280	260	254	254
query61	113	106	111	106
query62	760	660	679	660
query63	216	206	187	187
query64	1430	715	650	650
query65	3275	3150	3201	3150
query66	711	317	305	305
query67	15783	15438	15522	15438
query68	3850	593	577	577
query69	437	267	272	267
query70	1218	1107	1140	1107
query71	352	261	262	261
query72	6386	4201	4055	4055
query73	747	349	344	344
query74	10204	8984	8907	8907
query75	3369	2635	2655	2635
query76	1848	1020	1088	1020
query77	505	286	269	269
query78	10607	9530	9572	9530
query79	1199	594	613	594
query80	851	443	451	443
query81	516	246	237	237
query82	1266	115	114	114
query83	162	146	161	146
query84	287	83	77	77
query85	859	312	300	300
query86	345	292	301	292
query87	4386	4308	4224	4224
query88	3658	2426	2384	2384
query89	425	287	293	287
query90	1986	189	194	189
query91	182	147	148	147
query92	64	51	51	51
query93	1627	545	546	545
query94	754	290	298	290
query95	356	256	256	256
query96	615	285	288	285
query97	3357	3212	3209	3209
query98	216	202	197	197
query99	1586	1292	1281	1281
Total cold run time: 316903 ms
Total hot run time: 196914 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.21 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bb5cf73daf1ac8ca1f594a1c8436da2b3f07ace4, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.02
query3	0.23	0.07	0.07
query4	1.62	0.10	0.10
query5	0.53	0.51	0.52
query6	1.15	0.73	0.72
query7	0.02	0.01	0.01
query8	0.04	0.03	0.04
query9	0.55	0.52	0.51
query10	0.56	0.57	0.56
query11	0.14	0.10	0.09
query12	0.14	0.11	0.14
query13	0.61	0.59	0.60
query14	2.91	3.09	2.94
query15	0.90	0.84	0.83
query16	0.38	0.39	0.37
query17	1.05	1.07	1.06
query18	0.23	0.21	0.21
query19	1.88	1.91	1.92
query20	0.01	0.01	0.02
query21	15.37	0.58	0.58
query22	3.18	2.49	2.33
query23	17.07	0.91	0.85
query24	3.40	1.37	0.52
query25	0.14	0.18	0.08
query26	0.51	0.13	0.14
query27	0.05	0.04	0.04
query28	10.53	1.11	1.07
query29	12.61	3.26	3.24
query30	0.25	0.06	0.06
query31	2.87	0.38	0.38
query32	3.26	0.46	0.45
query33	2.92	3.01	3.03
query34	16.73	4.48	4.52
query35	4.54	4.62	4.50
query36	0.67	0.50	0.51
query37	0.10	0.06	0.06
query38	0.05	0.03	0.03
query39	0.03	0.02	0.02
query40	0.16	0.13	0.13
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 107.65 s
Total hot run time: 33.21 s

… Query Processing (apache#46310)

Problem Summary:
1. different values of inverted_index_max_expansions require separate
caches.
@zzzxl1993 zzzxl1993 force-pushed the branch-3.0.202401022034 branch from bb5cf73 to dbced92 Compare January 19, 2025 14:30
@zzzxl1993
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40658 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit dbced92662572863c79fe077bc1e43112dc49871, data reload: false

------ Round 1 ----------------------------------
q1	17633	7389	7255	7255
q2	2059	171	174	171
q3	10623	1075	1127	1075
q4	10557	747	853	747
q5	7731	2860	2758	2758
q6	235	151	145	145
q7	970	608	602	602
q8	9344	1963	1954	1954
q9	6570	6428	6408	6408
q10	7028	2290	2309	2290
q11	470	268	260	260
q12	394	216	212	212
q13	17847	3064	3010	3010
q14	228	205	223	205
q15	567	528	534	528
q16	708	613	624	613
q17	964	612	531	531
q18	7269	6710	6668	6668
q19	1416	1056	1027	1027
q20	462	209	201	201
q21	3980	3243	3050	3050
q22	1068	1016	948	948
Total cold run time: 108123 ms
Total hot run time: 40658 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7208	7173	7162	7162
q2	330	245	225	225
q3	2919	2937	2915	2915
q4	2036	1807	1803	1803
q5	5687	5752	5726	5726
q6	225	140	141	140
q7	2286	1768	1842	1768
q8	3359	3518	3560	3518
q9	8764	8876	8901	8876
q10	3576	3532	3566	3532
q11	593	504	509	504
q12	816	632	598	598
q13	8660	3258	3209	3209
q14	298	268	288	268
q15	581	533	529	529
q16	721	667	690	667
q17	1846	1619	1587	1587
q18	8203	7910	7628	7628
q19	1669	1490	1494	1490
q20	2108	1841	1887	1841
q21	5640	5377	5406	5377
q22	1202	1043	1043	1043
Total cold run time: 68727 ms
Total hot run time: 60406 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 198410 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit dbced92662572863c79fe077bc1e43112dc49871, data reload: false

query1	1317	951	936	936
query2	6229	2067	2059	2059
query3	10839	4257	4075	4075
query4	66476	28285	23510	23510
query5	5001	461	443	443
query6	414	182	175	175
query7	5523	315	312	312
query8	307	238	229	229
query9	8573	2698	2686	2686
query10	457	280	263	263
query11	17275	15238	15851	15238
query12	158	103	100	100
query13	1469	439	442	439
query14	10010	7906	7374	7374
query15	208	182	182	182
query16	7116	497	470	470
query17	1106	577	591	577
query18	1909	325	327	325
query19	208	165	166	165
query20	119	120	113	113
query21	214	103	111	103
query22	4527	4665	4619	4619
query23	34662	34216	34459	34216
query24	6122	2991	2987	2987
query25	534	429	414	414
query26	657	175	184	175
query27	1793	354	352	352
query28	3988	2497	2453	2453
query29	727	475	467	467
query30	247	169	169	169
query31	1031	858	842	842
query32	71	53	56	53
query33	466	303	308	303
query34	921	513	513	513
query35	890	718	728	718
query36	1082	967	970	967
query37	117	69	74	69
query38	4085	4113	3992	3992
query39	1524	1495	1482	1482
query40	217	99	101	99
query41	48	51	46	46
query42	117	102	100	100
query43	533	490	498	490
query44	1222	843	823	823
query45	189	168	169	168
query46	1153	751	745	745
query47	2071	1915	1934	1915
query48	482	383	390	383
query49	761	426	386	386
query50	853	423	433	423
query51	7457	7299	7325	7299
query52	99	89	90	89
query53	257	179	185	179
query54	564	475	456	456
query55	76	75	76	75
query56	260	252	242	242
query57	1211	1162	1108	1108
query58	212	206	222	206
query59	3102	2946	2890	2890
query60	281	240	247	240
query61	110	120	128	120
query62	777	657	662	657
query63	219	197	196	196
query64	1392	692	633	633
query65	3257	3216	3175	3175
query66	706	308	298	298
query67	15935	15668	15840	15668
query68	4010	581	581	581
query69	425	266	261	261
query70	1173	1138	1150	1138
query71	353	270	291	270
query72	6384	3958	4112	3958
query73	750	347	366	347
query74	10254	9019	9022	9019
query75	3389	2607	2643	2607
query76	1858	1076	1080	1076
query77	465	267	275	267
query78	10510	9659	9587	9587
query79	1513	610	599	599
query80	876	444	444	444
query81	526	247	242	242
query82	1289	120	112	112
query83	163	143	152	143
query84	277	82	74	74
query85	861	304	298	298
query86	358	269	306	269
query87	4490	4488	4335	4335
query88	3761	2413	2373	2373
query89	423	289	290	289
query90	2018	190	191	190
query91	180	146	151	146
query92	66	52	49	49
query93	1877	564	557	557
query94	786	302	288	288
query95	358	258	262	258
query96	615	285	288	285
query97	3342	3178	3222	3178
query98	209	208	203	203
query99	1639	1293	1298	1293
Total cold run time: 317131 ms
Total hot run time: 198410 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.49 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit dbced92662572863c79fe077bc1e43112dc49871, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.03
query3	0.23	0.07	0.06
query4	1.63	0.10	0.10
query5	0.52	0.52	0.52
query6	1.14	0.73	0.73
query7	0.02	0.01	0.01
query8	0.04	0.03	0.05
query9	0.56	0.51	0.49
query10	0.54	0.54	0.54
query11	0.13	0.10	0.10
query12	0.15	0.12	0.11
query13	0.61	0.60	0.58
query14	2.95	2.93	2.98
query15	0.93	0.83	0.82
query16	0.37	0.39	0.38
query17	0.98	1.04	1.02
query18	0.23	0.22	0.23
query19	1.94	1.84	1.92
query20	0.01	0.01	0.02
query21	15.35	0.60	0.60
query22	2.50	2.78	1.61
query23	17.11	1.02	0.87
query24	2.53	0.69	0.76
query25	0.28	0.22	0.06
query26	0.28	0.14	0.14
query27	0.04	0.04	0.06
query28	11.54	1.08	1.06
query29	12.62	3.25	3.27
query30	0.25	0.06	0.06
query31	2.86	0.39	0.39
query32	3.25	0.46	0.46
query33	3.03	2.99	2.99
query34	17.14	4.52	4.48
query35	4.56	4.52	4.48
query36	0.68	0.48	0.48
query37	0.10	0.06	0.05
query38	0.05	0.03	0.03
query39	0.03	0.03	0.02
query40	0.15	0.12	0.13
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.02
Total cold run time: 107.59 s
Total hot run time: 32.49 s

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 27, 2025
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@dataroaring dataroaring reopened this Feb 27, 2025
@dataroaring dataroaring merged commit 962fd55 into apache:branch-3.0 Feb 27, 2025
25 of 27 checks passed
@gavinchou gavinchou mentioned this pull request Apr 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants