Skip to content

Conversation

@morningman
Copy link
Contributor

part of #33449
in branch 2.0, I do a simple fix by using a separate thread pool for row count cache.
So that it won't block with other thread pool

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@morningman
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49612 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2f5b984d7de2b274e9be620877fcb5bad03d92cb, data reload: false

------ Round 1 ----------------------------------
q1	17674	4358	4323	4323
q2	2058	147	140	140
q3	10448	1879	1936	1879
q4	10360	1267	1336	1267
q5	8403	3960	3887	3887
q6	224	124	120	120
q7	2046	1603	1589	1589
q8	9488	2705	2706	2705
q9	11471	10620	10431	10431
q10	8610	3493	3502	3493
q11	407	234	235	234
q12	464	306	297	297
q13	18397	3950	3986	3950
q14	351	323	319	319
q15	496	451	461	451
q16	686	588	585	585
q17	1122	952	950	950
q18	7205	6750	7004	6750
q19	1672	1551	1527	1527
q20	528	303	293	293
q21	4444	4164	4022	4022
q22	493	400	403	400
Total cold run time: 117047 ms
Total hot run time: 49612 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4297	4328	4266	4266
q2	316	221	221	221
q3	4167	4102	4127	4102
q4	2734	2743	2721	2721
q5	7303	7200	7164	7164
q6	230	120	117	117
q7	3206	2811	2782	2782
q8	4332	4426	4455	4426
q9	16946	16919	16895	16895
q10	4213	4250	4232	4232
q11	740	680	684	680
q12	1014	842	826	826
q13	7150	3745	3697	3697
q14	452	424	422	422
q15	500	452	455	452
q16	748	715	689	689
q17	3814	3817	3773	3773
q18	8722	8765	8739	8739
q19	1689	1690	1674	1674
q20	2380	2158	2079	2079
q21	8517	8476	8550	8476
q22	1043	941	935	935
Total cold run time: 84513 ms
Total hot run time: 79368 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 201207 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2f5b984d7de2b274e9be620877fcb5bad03d92cb, data reload: false

query1	900	385	381	381
query2	6542	2158	2222	2158
query3	9622	200	206	200
query4	20324	18205	18066	18066
query5	21901	6632	6738	6632
query6	1396	224	284	224
query7	6227	291	298	291
query8	275	248	240	240
query9	3058	2666	2603	2603
query10	437	279	296	279
query11	11558	10749	10632	10632
query12	119	71	71	71
query13	5575	635	623	623
query14	18962	13652	13185	13185
query15	358	221	248	221
query16	6419	282	251	251
query17	1438	1423	857	857
query18	2293	403	401	401
query19	198	140	141	140
query20	70	75	74	74
query21	192	104	96	96
query22	5254	5103	5083	5083
query23	32215	31973	32016	31973
query24	6789	6506	6412	6412
query25	524	427	409	409
query26	518	154	158	154
query27	1739	292	286	286
query28	6107	2308	2243	2243
query29	2859	2602	2636	2602
query30	236	163	157	157
query31	928	721	712	712
query32	64	60	57	57
query33	390	255	253	253
query34	843	464	480	464
query35	1157	914	869	869
query36	1298	1239	1187	1187
query37	89	62	63	62
query38	3084	2961	2932	2932
query39	1360	1320	1310	1310
query40	197	95	99	95
query41	39	33	32	32
query42	83	85	82	82
query43	575	622	556	556
query44	1129	717	710	710
query45	246	227	230	227
query46	1257	975	976	975
query47	1921	1839	1742	1742
query48	988	661	651	651
query49	617	365	352	352
query50	859	621	621	621
query51	4708	4665	4717	4665
query52	95	76	92	76
query53	433	308	309	308
query54	2644	2454	2427	2427
query55	84	74	81	74
query56	218	209	185	185
query57	1200	1161	1061	1061
query58	213	203	194	194
query59	3339	3495	3246	3246
query60	202	194	213	194
query61	84	82	85	82
query62	850	565	494	494
query63	470	326	326	326
query64	2467	1459	1435	1435
query65	3645	3564	3563	3563
query66	753	391	349	349
query67	15422	15179	15269	15179
query68	8539	644	682	644
query69	567	330	338	330
query70	1864	1429	1456	1429
query71	416	298	305	298
query72	6445	3466	3461	3461
query73	734	323	315	315
query74	6291	5836	5818	5818
query75	4596	3737	3726	3726
query76	4749	1136	1221	1136
query77	632	246	244	244
query78	12540	12030	11617	11617
query79	11739	627	661	627
query80	1075	403	377	377
query81	489	230	222	222
query82	1268	101	90	90
query83	171	129	134	129
query84	253	69	67	67
query85	1001	282	278	278
query86	333	275	322	275
query87	3251	3058	3003	3003
query88	5092	2349	2321	2321
query89	486	287	294	287
query90	2052	204	205	204
query91	150	117	114	114
query92	54	51	51	51
query93	6346	545	550	545
query94	814	206	200	200
query95	1100	1053	1051	1051
query96	640	333	329	329
query97	6485	6311	6244	6244
query98	187	174	177	174
query99	3076	925	910	910
Total cold run time: 321491 ms
Total hot run time: 201207 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.11 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2f5b984d7de2b274e9be620877fcb5bad03d92cb, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.02	0.02
query3	0.24	0.04	0.04
query4	1.82	0.06	0.07
query5	0.53	0.53	0.51
query6	1.23	0.61	0.62
query7	0.02	0.01	0.01
query8	0.04	0.02	0.02
query9	0.54	0.48	0.48
query10	0.53	0.53	0.54
query11	0.12	0.08	0.09
query12	0.11	0.10	0.09
query13	0.61	0.60	0.60
query14	0.79	0.80	0.79
query15	0.77	0.75	0.78
query16	0.35	0.36	0.37
query17	0.98	0.98	1.01
query18	0.22	0.26	0.23
query19	1.93	1.78	1.89
query20	0.01	0.01	0.01
query21	15.46	0.59	0.56
query22	2.36	2.69	1.53
query23	17.08	1.07	0.91
query24	6.06	0.61	1.14
query25	0.36	0.13	0.04
query26	0.62	0.16	0.16
query27	0.06	0.03	0.05
query28	7.37	0.80	0.74
query29	12.62	2.34	2.24
query30	0.59	0.53	0.52
query31	2.79	0.39	0.38
query32	3.38	0.50	0.49
query33	3.06	3.11	3.07
query34	15.26	4.81	4.77
query35	4.85	4.84	4.84
query36	1.08	1.03	1.01
query37	0.06	0.04	0.05
query38	0.03	0.02	0.02
query39	0.02	0.01	0.01
query40	0.16	0.14	0.14
query41	0.06	0.02	0.01
query42	0.02	0.01	0.01
query43	0.02	0.02	0.01
Total cold run time: 104.3 s
Total hot run time: 30.11 s

@morningman morningman merged commit 2465159 into apache:branch-2.0 Apr 17, 2024
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
part of apache#33449
in branch 2.0, I do a simple fix by using a separate thread pool for row count cache.
So that it won't block with other thread pool
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants