Skip to content

Conversation

@dataroaring
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@dataroaring
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41075 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ec52ee4b22f145b60921ecc13de3ee93b8935d36, data reload: false

------ Round 1 ----------------------------------
q1	17592	7561	7393	7393
q2	2023	281	290	281
q3	12269	1056	1196	1056
q4	10579	764	793	764
q5	7737	2851	2828	2828
q6	235	152	148	148
q7	1012	633	626	626
q8	9355	1920	1949	1920
q9	6565	6443	6343	6343
q10	6981	2315	2332	2315
q11	445	260	268	260
q12	416	227	223	223
q13	17770	2988	2980	2980
q14	241	204	213	204
q15	569	530	530	530
q16	663	572	575	572
q17	966	550	602	550
q18	7237	6699	6807	6699
q19	1345	1033	971	971
q20	488	202	205	202
q21	3925	3210	3297	3210
q22	1112	1000	1009	1000
Total cold run time: 109525 ms
Total hot run time: 41075 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7185	7214	7186	7186
q2	324	230	230	230
q3	2991	2924	2961	2924
q4	2086	1906	1831	1831
q5	5735	5775	5701	5701
q6	229	147	143	143
q7	2299	1813	1772	1772
q8	3352	3562	3434	3434
q9	8980	8889	8825	8825
q10	3622	3530	3498	3498
q11	612	509	497	497
q12	875	629	636	629
q13	11097	3176	3168	3168
q14	307	271	291	271
q15	573	535	509	509
q16	696	631	633	631
q17	1819	1607	1613	1607
q18	8215	7913	7439	7439
q19	1689	1462	1347	1347
q20	2114	1865	1867	1865
q21	5584	5411	5379	5379
q22	1152	1041	1026	1026
Total cold run time: 71536 ms
Total hot run time: 59912 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191065 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ec52ee4b22f145b60921ecc13de3ee93b8935d36, data reload: false

query1	863	388	402	388
query2	6252	2067	1964	1964
query3	8683	191	200	191
query4	33735	23562	23486	23486
query5	3473	459	449	449
query6	265	160	159	159
query7	4189	319	304	304
query8	284	265	224	224
query9	9302	2649	2647	2647
query10	459	283	282	282
query11	17881	15126	15086	15086
query12	154	99	98	98
query13	1598	478	443	443
query14	9786	6512	7539	6512
query15	243	165	175	165
query16	7996	458	451	451
query17	1645	617	617	617
query18	2106	359	311	311
query19	331	155	153	153
query20	119	111	111	111
query21	215	107	106	106
query22	4856	4567	4595	4567
query23	34912	33997	34002	33997
query24	11039	2833	2841	2833
query25	623	371	375	371
query26	1215	158	158	158
query27	2639	300	299	299
query28	7888	2423	2416	2416
query29	814	420	420	420
query30	256	155	153	153
query31	1012	783	800	783
query32	93	49	54	49
query33	756	293	289	289
query34	932	501	512	501
query35	889	752	735	735
query36	1097	938	967	938
query37	146	92	82	82
query38	3967	4006	3895	3895
query39	1477	1421	1449	1421
query40	204	92	92	92
query41	45	41	39	39
query42	109	91	92	91
query43	512	481	488	481
query44	1260	766	793	766
query45	194	161	160	160
query46	1148	701	712	701
query47	1958	1802	1847	1802
query48	418	331	333	331
query49	931	399	378	378
query50	827	404	409	404
query51	7077	6926	6933	6926
query52	93	82	84	82
query53	256	174	176	174
query54	1121	465	459	459
query55	78	72	74	72
query56	273	256	248	248
query57	1267	1166	1137	1137
query58	229	239	241	239
query59	3245	2983	2966	2966
query60	294	268	270	268
query61	101	101	105	101
query62	865	664	657	657
query63	220	185	189	185
query64	4003	636	630	630
query65	3235	3174	3172	3172
query66	860	316	306	306
query67	15962	15626	15544	15544
query68	3068	588	580	580
query69	437	306	307	306
query70	1207	1126	1110	1110
query71	339	272	275	272
query72	6186	3947	4089	3947
query73	761	345	350	345
query74	9464	9040	9017	9017
query75	3416	2708	2676	2676
query76	1938	942	1036	942
query77	425	285	284	284
query78	10535	9635	9587	9587
query79	2926	580	609	580
query80	2363	445	427	427
query81	602	241	243	241
query82	674	142	139	139
query83	281	136	133	133
query84	259	75	74	74
query85	1665	290	279	279
query86	439	313	298	298
query87	4470	4313	4310	4310
query88	3814	2425	2393	2393
query89	413	285	288	285
query90	1965	189	188	188
query91	138	107	108	107
query92	68	49	47	47
query93	2399	556	555	555
query94	956	293	255	255
query95	368	256	252	252
query96	630	279	280	279
query97	3350	3197	3248	3197
query98	208	201	199	199
query99	1615	1310	1296	1296
Total cold run time: 298354 ms
Total hot run time: 191065 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.56 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ec52ee4b22f145b60921ecc13de3ee93b8935d36, data reload: false

query1	0.05	0.04	0.04
query2	0.06	0.03	0.03
query3	0.22	0.07	0.07
query4	1.64	0.11	0.10
query5	0.54	0.52	0.50
query6	1.14	0.73	0.71
query7	0.02	0.01	0.02
query8	0.04	0.03	0.04
query9	0.56	0.50	0.50
query10	0.54	0.53	0.54
query11	0.14	0.10	0.11
query12	0.14	0.11	0.11
query13	0.62	0.60	0.59
query14	2.73	2.73	2.80
query15	0.90	0.81	0.82
query16	0.38	0.39	0.38
query17	1.07	1.07	1.00
query18	0.20	0.20	0.19
query19	1.87	1.86	1.99
query20	0.01	0.01	0.01
query21	15.37	0.59	0.59
query22	3.02	2.15	2.21
query23	16.83	0.99	0.87
query24	2.83	1.06	0.66
query25	0.17	0.11	0.04
query26	0.52	0.13	0.14
query27	0.04	0.04	0.03
query28	11.28	1.10	1.07
query29	12.59	3.19	3.18
query30	0.25	0.06	0.06
query31	2.86	0.37	0.38
query32	3.27	0.48	0.46
query33	2.96	3.01	2.96
query34	16.89	4.47	4.46
query35	4.51	4.45	4.55
query36	0.69	0.47	0.49
query37	0.08	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.15	0.12	0.13
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 107.4 s
Total hot run time: 32.56 s

airborne12
airborne12 previously approved these changes Oct 6, 2024
Copy link
Member

@airborne12 airborne12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Oct 6, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Oct 6, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Oct 6, 2024

PR approved by anyone and no changes requested.

morrySnow
morrySnow previously approved these changes Oct 8, 2024
@dataroaring dataroaring dismissed stale reviews from morrySnow and airborne12 via f66857b October 8, 2024 03:16
@dataroaring
Copy link
Contributor Author

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Oct 8, 2024
@doris-robot
Copy link

TPC-H: Total hot run time: 40942 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f66857b6450450596fcba57ec91daad31da420ae, data reload: false

------ Round 1 ----------------------------------
q1	17613	8208	7230	7230
q2	2026	277	270	270
q3	12218	1065	1164	1065
q4	10582	712	754	712
q5	7748	2908	2833	2833
q6	239	154	153	153
q7	1038	650	636	636
q8	9366	1941	2007	1941
q9	6540	6397	6409	6397
q10	7009	2297	2344	2297
q11	471	252	251	251
q12	428	221	223	221
q13	17765	3024	3056	3024
q14	236	211	218	211
q15	575	531	523	523
q16	629	584	596	584
q17	987	526	597	526
q18	7342	6714	6736	6714
q19	1357	984	892	892
q20	473	206	202	202
q21	4232	3304	3263	3263
q22	1108	1009	997	997
Total cold run time: 109982 ms
Total hot run time: 40942 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7302	7309	7363	7309
q2	328	231	230	230
q3	3122	2987	2984	2984
q4	2063	1843	1802	1802
q5	5866	5972	5949	5949
q6	238	154	148	148
q7	2370	1962	1944	1944
q8	3473	3545	3597	3545
q9	9350	9145	9068	9068
q10	3656	3571	3571	3571
q11	603	488	501	488
q12	820	639	626	626
q13	15219	3175	3194	3175
q14	310	272	277	272
q15	589	526	534	526
q16	693	661	645	645
q17	1813	1603	1645	1603
q18	8326	7734	7644	7644
q19	1696	1392	1528	1392
q20	2091	1889	1866	1866
q21	5432	5368	5488	5368
q22	1139	1077	1087	1077
Total cold run time: 76499 ms
Total hot run time: 61232 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191903 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f66857b6450450596fcba57ec91daad31da420ae, data reload: false

query1	904	392	414	392
query2	6251	2048	2044	2044
query3	8674	199	205	199
query4	33895	23577	23719	23577
query5	3436	468	456	456
query6	253	161	162	161
query7	4197	313	310	310
query8	271	211	223	211
query9	9376	2683	2698	2683
query10	472	300	283	283
query11	17811	15086	15266	15086
query12	155	98	96	96
query13	1594	455	445	445
query14	9638	7470	7546	7470
query15	268	171	179	171
query16	8073	476	469	469
query17	1784	610	603	603
query18	2155	328	334	328
query19	376	160	157	157
query20	125	127	114	114
query21	216	104	106	104
query22	4804	4648	4535	4535
query23	34961	34035	34005	34005
query24	11020	2834	2904	2834
query25	620	416	411	411
query26	1104	169	163	163
query27	2257	305	308	305
query28	7405	2403	2408	2403
query29	820	450	433	433
query30	258	153	152	152
query31	1035	781	789	781
query32	103	55	56	55
query33	779	301	294	294
query34	910	501	519	501
query35	897	725	707	707
query36	1115	964	968	964
query37	152	98	90	90
query38	4007	3840	3997	3840
query39	1475	1433	1436	1433
query40	203	100	95	95
query41	48	44	45	44
query42	116	92	93	92
query43	541	488	493	488
query44	1204	806	817	806
query45	199	164	164	164
query46	1157	704	716	704
query47	1941	1792	1818	1792
query48	443	353	344	344
query49	877	415	396	396
query50	827	424	406	406
query51	7133	7005	6942	6942
query52	105	85	88	85
query53	258	177	179	177
query54	1236	474	469	469
query55	77	79	80	79
query56	297	285	284	284
query57	1243	1122	1134	1122
query58	220	241	225	225
query59	3140	2905	3125	2905
query60	283	266	258	258
query61	106	100	106	100
query62	847	702	644	644
query63	218	187	179	179
query64	3901	636	610	610
query65	3237	3211	3184	3184
query66	767	308	308	308
query67	15968	15575	15719	15575
query68	4391	570	551	551
query69	526	318	293	293
query70	1185	1123	1075	1075
query71	343	279	274	274
query72	7505	4054	3909	3909
query73	787	342	343	342
query74	10269	9042	8900	8900
query75	3389	2702	2662	2662
query76	3077	884	945	884
query77	610	301	297	297
query78	10566	9509	9486	9486
query79	1860	600	594	594
query80	946	467	450	450
query81	555	243	240	240
query82	369	141	143	141
query83	304	134	140	134
query84	271	80	84	80
query85	1156	292	292	292
query86	337	318	296	296
query87	4413	4275	4262	4262
query88	3600	2428	2383	2383
query89	391	286	291	286
query90	2179	189	182	182
query91	144	108	108	108
query92	61	51	49	49
query93	1071	541	543	541
query94	897	308	292	292
query95	349	247	257	247
query96	611	277	284	277
query97	3346	3133	3182	3133
query98	219	207	198	198
query99	1543	1307	1303	1303
Total cold run time: 297508 ms
Total hot run time: 191903 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.93 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f66857b6450450596fcba57ec91daad31da420ae, data reload: false

query1	0.04	0.05	0.04
query2	0.07	0.03	0.02
query3	0.22	0.06	0.06
query4	1.65	0.10	0.10
query5	0.54	0.49	0.51
query6	1.14	0.73	0.74
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.55	0.51	0.49
query10	0.56	0.56	0.56
query11	0.14	0.11	0.11
query12	0.14	0.11	0.12
query13	0.61	0.59	0.60
query14	2.78	2.74	2.84
query15	0.90	0.84	0.82
query16	0.37	0.38	0.37
query17	1.05	1.06	1.05
query18	0.20	0.19	0.19
query19	1.97	1.88	2.04
query20	0.01	0.01	0.02
query21	15.35	0.58	0.58
query22	2.61	2.61	2.08
query23	16.94	0.95	0.94
query24	2.64	0.95	0.76
query25	0.32	0.07	0.06
query26	0.42	0.15	0.13
query27	0.04	0.04	0.04
query28	11.23	1.10	1.06
query29	12.58	3.29	3.28
query30	0.24	0.05	0.07
query31	2.88	0.37	0.37
query32	3.31	0.46	0.46
query33	2.96	2.98	3.06
query34	16.94	4.44	4.49
query35	4.52	4.51	4.56
query36	0.67	0.48	0.50
query37	0.08	0.06	0.06
query38	0.04	0.03	0.04
query39	0.03	0.03	0.02
query40	0.16	0.12	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 107.1 s
Total hot run time: 32.93 s

dataroaring and others added 2 commits October 8, 2024 20:12
…tSpotStmt.java

Co-authored-by: morrySnow <101034200+morrySnow@users.noreply.github.com>
@dataroaring dataroaring force-pushed the show_cache_hotspot_fix branch from f66857b to 7e2966f Compare October 8, 2024 12:12
@dataroaring
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40487 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7e2966f4e2f72cb7d9bdb836338fda4e81724e63, data reload: false

------ Round 1 ----------------------------------
q1	17565	7379	7229	7229
q2	2013	274	263	263
q3	12099	1053	1238	1053
q4	10570	735	724	724
q5	7737	2811	2771	2771
q6	235	151	149	149
q7	998	651	632	632
q8	9359	1886	1945	1886
q9	6553	6419	6379	6379
q10	6992	2272	2297	2272
q11	448	246	244	244
q12	416	217	215	215
q13	17782	2956	2950	2950
q14	246	216	221	216
q15	564	543	521	521
q16	655	598	597	597
q17	964	535	549	535
q18	7200	6784	6689	6689
q19	1353	949	862	862
q20	494	209	198	198
q21	3984	3117	3164	3117
q22	1118	985	1000	985
Total cold run time: 109345 ms
Total hot run time: 40487 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7251	7221	7264	7221
q2	322	236	230	230
q3	2924	2890	2936	2890
q4	2105	1842	1817	1817
q5	5708	5734	6023	5734
q6	237	142	142	142
q7	2252	1826	1819	1819
q8	3380	3550	3471	3471
q9	8861	8837	8867	8837
q10	3616	3544	3525	3525
q11	574	478	489	478
q12	855	609	666	609
q13	11345	3180	3129	3129
q14	301	262	278	262
q15	578	513	553	513
q16	710	656	637	637
q17	1854	1625	1583	1583
q18	8276	7744	7633	7633
q19	1701	1641	1657	1641
q20	2133	1869	1880	1869
q21	5524	5244	5266	5244
q22	1115	1023	1037	1023
Total cold run time: 71622 ms
Total hot run time: 60307 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192272 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7e2966f4e2f72cb7d9bdb836338fda4e81724e63, data reload: false

query1	942	392	408	392
query2	6268	2111	2028	2028
query3	8684	195	197	195
query4	34044	23518	23507	23507
query5	3849	461	461	461
query6	282	169	167	167
query7	4191	322	318	318
query8	285	224	220	220
query9	9500	2719	2699	2699
query10	440	272	272	272
query11	17977	15194	15245	15194
query12	156	101	99	99
query13	1574	463	451	451
query14	9682	7006	7357	7006
query15	255	170	172	170
query16	8004	477	412	412
query17	1678	640	599	599
query18	2159	323	323	323
query19	372	165	157	157
query20	119	116	116	116
query21	214	106	111	106
query22	4732	4598	4609	4598
query23	34772	34220	34009	34009
query24	10955	2841	2808	2808
query25	608	421	408	408
query26	1199	163	161	161
query27	2598	300	311	300
query28	7784	2397	2408	2397
query29	787	436	426	426
query30	252	156	156	156
query31	1028	807	828	807
query32	104	54	57	54
query33	762	308	298	298
query34	901	499	511	499
query35	871	736	710	710
query36	1108	962	951	951
query37	148	100	87	87
query38	4127	3914	3983	3914
query39	1473	1467	1550	1467
query40	212	96	97	96
query41	47	44	44	44
query42	117	96	96	96
query43	560	496	494	494
query44	1283	794	811	794
query45	200	170	172	170
query46	1130	710	702	702
query47	1937	1816	1841	1816
query48	444	359	348	348
query49	942	419	398	398
query50	828	421	421	421
query51	7206	7058	7056	7056
query52	102	87	88	87
query53	252	185	180	180
query54	1279	487	480	480
query55	79	76	75	75
query56	279	269	281	269
query57	1263	1158	1159	1158
query58	238	249	246	246
query59	3157	3174	2934	2934
query60	300	272	259	259
query61	107	101	106	101
query62	870	660	665	660
query63	211	184	184	184
query64	5053	660	618	618
query65	3229	3196	3216	3196
query66	1399	315	306	306
query67	15784	15686	15776	15686
query68	4380	568	563	563
query69	444	289	327	289
query70	1154	1203	1146	1146
query71	339	275	270	270
query72	7077	3952	3966	3952
query73	774	344	355	344
query74	10253	9139	9025	9025
query75	3434	2713	2671	2671
query76	2608	898	937	898
query77	584	298	303	298
query78	10405	9604	9471	9471
query79	1840	591	594	591
query80	1038	447	462	447
query81	589	235	241	235
query82	717	145	136	136
query83	292	145	136	136
query84	279	78	77	77
query85	1367	307	302	302
query86	377	289	304	289
query87	4433	4448	4261	4261
query88	3171	2474	2437	2437
query89	395	285	286	285
query90	1953	188	182	182
query91	136	107	117	107
query92	58	47	46	46
query93	1810	542	547	542
query94	974	303	293	293
query95	350	259	257	257
query96	612	288	288	288
query97	3317	3134	3186	3134
query98	226	202	209	202
query99	1509	1319	1282	1282
Total cold run time: 300287 ms
Total hot run time: 192272 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.13 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7e2966f4e2f72cb7d9bdb836338fda4e81724e63, data reload: false

query1	0.05	0.04	0.04
query2	0.06	0.03	0.02
query3	0.23	0.06	0.07
query4	1.64	0.10	0.10
query5	0.53	0.53	0.52
query6	1.13	0.72	0.72
query7	0.02	0.02	0.02
query8	0.04	0.03	0.04
query9	0.56	0.51	0.50
query10	0.57	0.57	0.54
query11	0.14	0.10	0.10
query12	0.14	0.11	0.11
query13	0.62	0.61	0.59
query14	2.86	2.72	2.81
query15	0.90	0.83	0.83
query16	0.39	0.37	0.37
query17	1.04	1.05	1.00
query18	0.19	0.20	0.20
query19	1.95	1.81	2.02
query20	0.02	0.01	0.01
query21	15.35	0.61	0.62
query22	2.57	2.70	1.59
query23	16.82	0.80	0.81
query24	2.92	0.59	1.06
query25	0.27	0.15	0.13
query26	0.31	0.15	0.13
query27	0.04	0.04	0.04
query28	11.10	1.11	1.07
query29	12.53	3.28	3.28
query30	0.25	0.06	0.06
query31	2.88	0.38	0.39
query32	3.28	0.49	0.47
query33	3.00	3.04	3.00
query34	16.99	4.45	4.45
query35	4.47	4.47	4.45
query36	0.67	0.51	0.49
query37	0.09	0.06	0.06
query38	0.04	0.04	0.03
query39	0.03	0.02	0.02
query40	0.16	0.13	0.14
query41	0.07	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.02
Total cold run time: 106.99 s
Total hot run time: 32.13 s

@github-actions
Copy link
Contributor

github-actions bot commented Oct 8, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Oct 8, 2024
Copy link
Member

@airborne12 airborne12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit 047fa26 into apache:master Oct 10, 2024
eldenmoon pushed a commit to eldenmoon/incubator-doris that referenced this pull request Oct 10, 2024
cjj2010 pushed a commit to cjj2010/doris that referenced this pull request Oct 12, 2024
amorynan pushed a commit to amorynan/doris that referenced this pull request Oct 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.2-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants