[branch-2.0][improvement](mysql catalog) disable mysql AbandonedConnectionCleanup Thread #36942

zy-kkk · 2024-06-27T11:11:10Z

When using mysql catalog, mysql jdbc driver will generate an AbandonedConnectionCleanupThread for each database connection. This is a virtual reference, which will accumulate over time as database connections are constantly created, eventually causing OOM. Therefore, in our usage scenario, we need to turn off this thread because our database connection recycling depends on the connection pool. But please note that this switch is only for MySQL JDBC Driver versions greater than 8.0.22

zy-kkk · 2024-06-27T11:11:18Z

run buildall

doris-robot · 2024-06-27T11:11:20Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

doris-robot · 2024-06-27T11:54:50Z

TPC-H: Total hot run time: 50039 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3f2a70ac6c9bb11505fb787e11b3d7d3352728f7, data reload: false

------ Round 1 ----------------------------------
q1	17897	4353	4340	4340
q2	2040	156	151	151
q3	10421	1881	1944	1881
q4	10312	1291	1305	1291
q5	8458	3924	3864	3864
q6	234	126	126	126
q7	2033	1590	1599	1590
q8	9288	2717	2689	2689
q9	10961	10599	10489	10489
q10	8656	3523	3513	3513
q11	423	247	255	247
q12	471	310	316	310
q13	18351	3983	4028	3983
q14	351	317	326	317
q15	503	463	461	461
q16	680	569	571	569
q17	1132	949	974	949
q18	7266	6875	6856	6856
q19	1751	1661	1600	1600
q20	560	305	308	305
q21	4414	4092	4074	4074
q22	528	434	451	434
Total cold run time: 116730 ms
Total hot run time: 50039 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4349	4269	4286	4269
q2	323	233	232	232
q3	4166	4126	4138	4126
q4	2771	2723	2738	2723
q5	7227	7099	7041	7041
q6	238	121	120	120
q7	3286	2879	2836	2836
q8	4354	4485	4471	4471
q9	17335	16744	17001	16744
q10	4230	4293	4261	4261
q11	734	705	674	674
q12	1053	859	856	856
q13	6696	3703	3732	3703
q14	443	428	418	418
q15	507	458	464	458
q16	736	693	691	691
q17	3840	3900	3835	3835
q18	8647	8704	8756	8704
q19	1713	1700	1700	1700
q20	2375	2129	2063	2063
q21	8408	8421	8495	8421
q22	1111	1067	1057	1057
Total cold run time: 84542 ms
Total hot run time: 79403 ms

doris-robot · 2024-06-27T12:07:31Z

TPC-DS: Total hot run time: 202971 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3f2a70ac6c9bb11505fb787e11b3d7d3352728f7, data reload: false

query1	952	431	383	383
query2	7111	2566	2625	2566
query3	6919	207	203	203
query4	20388	17869	17813	17813
query5	19726	6514	6563	6514
query6	320	214	230	214
query7	4298	307	326	307
query8	429	418	420	418
query9	3201	2708	2631	2631
query10	420	315	281	281
query11	11257	10685	10788	10685
query12	129	79	75	75
query13	5608	703	685	685
query14	18693	13693	13008	13008
query15	365	248	251	248
query16	6443	287	265	265
query17	1486	1503	891	891
query18	2253	420	413	413
query19	210	151	151	151
query20	83	80	82	80
query21	197	101	107	101
query22	5321	5152	5048	5048
query23	32546	32131	31884	31884
query24	6813	6624	6503	6503
query25	538	462	425	425
query26	526	176	162	162
query27	1831	294	309	294
query28	6083	2383	2338	2338
query29	2848	2719	2791	2719
query30	244	164	169	164
query31	900	726	745	726
query32	81	64	63	63
query33	405	253	250	250
query34	873	468	481	468
query35	1120	924	949	924
query36	1326	1145	1124	1124
query37	94	60	61	60
query38	3129	2963	2919	2919
query39	1371	1348	1329	1329
query40	204	96	100	96
query41	46	45	42	42
query42	90	86	91	86
query43	754	698	695	695
query44	1164	720	723	720
query45	249	243	241	241
query46	1248	994	989	989
query47	1978	1698	1817	1698
query48	1026	729	731	729
query49	626	386	378	378
query50	872	635	581	581
query51	4750	4726	4668	4668
query52	90	98	84	84
query53	455	335	326	326
query54	2655	2461	2458	2458
query55	87	85	86	85
query56	243	220	224	220
query57	1235	1152	1075	1075
query58	222	220	216	216
query59	4543	3762	3970	3762
query60	211	207	224	207
query61	101	99	101	99
query62	822	491	452	452
query63	496	343	358	343
query64	2448	1555	1449	1449
query65	3648	3573	3625	3573
query66	813	396	385	385
query67	15445	15977	15223	15223
query68	10576	628	649	628
query69	584	336	354	336
query70	2225	1199	1352	1199
query71	424	307	324	307
query72	6516	3541	3551	3541
query73	737	327	312	312
query74	6341	5869	5863	5863
query75	5522	3684	3706	3684
query76	6851	1134	1211	1134
query77	1185	257	254	254
query78	12922	11739	14819	11739
query79	7003	634	632	632
query80	810	419	415	415
query81	494	237	237	237
query82	492	97	94	94
query83	171	137	132	132
query84	263	72	72	72
query85	876	323	325	323
query86	336	298	306	298
query87	3201	2990	3012	2990
query88	3866	2278	2293	2278
query89	343	286	304	286
query90	1976	206	213	206
query91	177	141	139	139
query92	60	56	54	54
query93	3067	546	552	546
query94	711	209	205	205
query95	1113	1087	1077	1077
query96	640	328	318	318
query97	6514	6501	6444	6444
query98	195	170	178	170
query99	2861	847	846	846
Total cold run time: 312110 ms
Total hot run time: 202971 ms

doris-robot · 2024-06-27T12:12:56Z

ClickBench: Total hot run time: 31.42 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3f2a70ac6c9bb11505fb787e11b3d7d3352728f7, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.02	0.02
query3	0.25	0.04	0.06
query4	1.75	0.10	0.10
query5	0.53	0.52	0.52
query6	1.28	0.62	0.61
query7	0.02	0.01	0.01
query8	0.03	0.02	0.02
query9	0.52	0.48	0.48
query10	0.53	0.54	0.53
query11	0.12	0.08	0.09
query12	0.12	0.09	0.09
query13	0.63	0.62	0.62
query14	0.78	0.78	0.79
query15	0.77	0.75	0.78
query16	0.36	0.35	0.35
query17	1.00	1.01	1.02
query18	0.21	0.25	0.25
query19	1.92	1.88	1.77
query20	0.02	0.01	0.01
query21	15.46	0.57	0.54
query22	2.04	2.49	1.57
query23	16.94	0.86	1.12
query24	6.07	1.92	2.00
query25	0.35	0.10	0.05
query26	0.86	0.16	0.18
query27	0.04	0.04	0.04
query28	5.42	0.78	0.73
query29	12.64	2.13	2.28
query30	0.55	0.52	0.51
query31	2.80	0.39	0.37
query32	3.37	0.50	0.50
query33	3.07	3.04	3.06
query34	15.26	4.86	4.80
query35	4.82	4.88	4.88
query36	1.06	1.01	1.01
query37	0.06	0.05	0.05
query38	0.04	0.02	0.02
query39	0.02	0.02	0.01
query40	0.17	0.14	0.14
query41	0.06	0.02	0.01
query42	0.02	0.01	0.02
query43	0.03	0.02	0.01
Total cold run time: 102.08 s
Total hot run time: 31.42 s

doris-robot · 2024-06-27T12:16:10Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 3f2a70ac6c9bb11505fb787e11b3d7d3352728f7 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.4 seconds inserted 10000000 Rows, about 467K ops/s

… Thread (apache#36655) When using mysql catalog, mysql jdbc driver will generate an `AbandonedConnectionCleanupThread` for each database connection. This is a virtual reference, which will accumulate over time as database connections are constantly created, eventually causing OOM. Therefore, in our usage scenario, we need to turn off this thread because our database connection recycling depends on the connection pool. But please note that this switch is only for MySQL JDBC Driver versions greater than 8.0.22

morningman · 2024-06-29T10:36:32Z

run buildall

doris-robot · 2024-06-29T11:08:13Z

TPC-H: Total hot run time: 50550 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4321fae75262e0099fd96d3ff53c3209ff3d11d7, data reload: false

------ Round 1 ----------------------------------
q1	18124	4442	4398	4398
q2	2073	155	147	147
q3	10412	1918	1944	1918
q4	10601	1241	1365	1241
q5	9245	3910	3925	3910
q6	228	149	128	128
q7	2060	1578	1600	1578
q8	9520	2741	2721	2721
q9	11037	10856	10716	10716
q10	8933	3702	3702	3702
q11	434	263	264	263
q12	480	304	310	304
q13	19772	4006	4042	4006
q14	378	338	328	328
q15	524	480	469	469
q16	818	595	648	595
q17	1524	976	946	946
q18	7316	6898	6754	6754
q19	1768	1608	1571	1571
q20	545	307	316	307
q21	4371	4115	4146	4115
q22	544	446	433	433
Total cold run time: 120707 ms
Total hot run time: 50550 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4263	4267	4281	4267
q2	321	225	218	218
q3	4152	4166	4110	4110
q4	2742	2729	2734	2729
q5	7212	7132	7108	7108
q6	236	121	118	118
q7	3209	2755	2826	2755
q8	4315	4430	4425	4425
q9	17366	17216	17299	17216
q10	4366	4310	4226	4226
q11	762	675	678	675
q12	1048	850	862	850
q13	6998	3720	3738	3720
q14	447	424	413	413
q15	509	465	463	463
q16	742	683	675	675
q17	3874	3784	3795	3784
q18	8750	8607	8735	8607
q19	1704	1701	1639	1639
q20	2372	2082	2090	2082
q21	8468	8494	8424	8424
q22	1073	981	951	951
Total cold run time: 84929 ms
Total hot run time: 79455 ms

doris-robot · 2024-06-29T11:20:51Z

TPC-DS: Total hot run time: 203407 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4321fae75262e0099fd96d3ff53c3209ff3d11d7, data reload: false

query1	936	384	427	384
query2	6551	2696	2603	2603
query3	6931	204	207	204
query4	20778	17984	17941	17941
query5	19746	6498	6535	6498
query6	320	224	231	224
query7	4389	294	295	294
query8	440	421	407	407
query9	3113	2678	2605	2605
query10	424	311	312	311
query11	11398	10623	10644	10623
query12	119	72	74	72
query13	5591	691	664	664
query14	17997	13179	13482	13179
query15	357	245	238	238
query16	6447	293	261	261
query17	1412	1538	858	858
query18	2290	408	417	408
query19	212	144	150	144
query20	78	78	79	78
query21	193	95	92	92
query22	5149	5068	5026	5026
query23	32492	31907	31863	31863
query24	6733	6511	6502	6502
query25	531	440	417	417
query26	519	161	157	157
query27	1839	287	292	287
query28	6090	2370	2308	2308
query29	2813	2693	2733	2693
query30	245	165	167	165
query31	896	729	730	729
query32	70	63	62	62
query33	403	244	260	244
query34	859	456	476	456
query35	1118	919	891	891
query36	1352	1126	1220	1126
query37	97	59	59	59
query38	3114	2900	2931	2900
query39	1385	1345	1336	1336
query40	210	98	97	97
query41	46	44	44	44
query42	80	80	77	77
query43	691	702	787	702
query44	1129	709	711	709
query45	254	234	235	234
query46	1219	967	963	963
query47	2028	1719	1906	1719
query48	1024	692	693	692
query49	623	358	378	358
query50	867	578	565	565
query51	4882	4702	4623	4623
query52	88	90	85	85
query53	446	313	323	313
query54	2667	2436	2426	2426
query55	90	79	83	79
query56	231	221	204	204
query57	1296	1095	1134	1095
query58	218	202	207	202
query59	4047	4046	3816	3816
query60	216	189	214	189
query61	100	99	95	95
query62	850	485	466	466
query63	476	334	337	334
query64	2507	1554	1508	1508
query65	3625	3560	3535	3535
query66	763	383	369	369
query67	15568	15826	15807	15807
query68	9335	626	661	626
query69	567	342	358	342
query70	1771	1309	1391	1309
query71	428	306	312	306
query72	6659	3546	3486	3486
query73	739	327	329	327
query74	6358	5865	5836	5836
query75	5416	3721	3726	3721
query76	5894	1129	1195	1129
query77	1014	258	268	258
query78	12642	11764	11685	11685
query79	6785	632	637	632
query80	956	406	408	406
query81	486	234	235	234
query82	1533	102	98	98
query83	178	142	135	135
query84	261	74	73	73
query85	880	330	328	328
query86	332	302	312	302
query87	3233	3008	2999	2999
query88	4592	2341	2334	2334
query89	373	328	308	308
query90	1974	214	218	214
query91	172	144	140	140
query92	58	53	54	53
query93	4068	573	583	573
query94	715	210	216	210
query95	1096	1067	1055	1055
query96	645	327	329	327
query97	6499	6343	6342	6342
query98	197	173	167	167
query99	2956	837	980	837
Total cold run time: 310480 ms
Total hot run time: 203407 ms

doris-robot · 2024-06-29T11:26:16Z

ClickBench: Total hot run time: 31.4 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4321fae75262e0099fd96d3ff53c3209ff3d11d7, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.02	0.02
query3	0.25	0.05	0.04
query4	1.81	0.06	0.05
query5	0.53	0.52	0.52
query6	1.27	0.62	0.62
query7	0.01	0.01	0.01
query8	0.03	0.03	0.02
query9	0.52	0.50	0.48
query10	0.54	0.53	0.53
query11	0.11	0.10	0.09
query12	0.12	0.10	0.10
query13	0.61	0.61	0.62
query14	0.79	0.78	0.78
query15	0.79	0.77	0.78
query16	0.37	0.37	0.37
query17	1.02	1.02	1.03
query18	0.24	0.23	0.27
query19	1.91	1.80	1.87
query20	0.01	0.01	0.02
query21	15.47	0.55	0.55
query22	2.02	2.16	1.63
query23	17.13	1.12	1.05
query24	4.23	1.40	1.87
query25	0.37	0.08	0.05
query26	0.66	0.17	0.17
query27	0.05	0.05	0.03
query28	7.60	0.77	0.71
query29	12.59	2.34	2.35
query30	0.55	0.51	0.52
query31	2.80	0.39	0.37
query32	3.39	0.49	0.49
query33	3.05	3.13	3.08
query34	15.26	4.82	4.81
query35	4.90	4.86	4.87
query36	1.06	1.02	1.01
query37	0.06	0.05	0.04
query38	0.03	0.02	0.02
query39	0.02	0.02	0.02
query40	0.15	0.15	0.14
query41	0.06	0.01	0.02
query42	0.02	0.02	0.01
query43	0.02	0.01	0.02
Total cold run time: 102.51 s
Total hot run time: 31.4 s

doris-robot · 2024-06-29T11:29:28Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 4321fae75262e0099fd96d3ff53c3209ff3d11d7 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.4 seconds inserted 10000000 Rows, about 467K ops/s

zy-kkk · 2024-07-01T07:05:47Z

run buildall

doris-robot · 2024-07-01T07:36:51Z

TPC-H: Total hot run time: 49940 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4321fae75262e0099fd96d3ff53c3209ff3d11d7, data reload: false

------ Round 1 ----------------------------------
q1	17750	4393	4372	4372
q2	2062	154	146	146
q3	10268	1887	1919	1887
q4	10322	1208	1305	1208
q5	8587	3929	3937	3929
q6	236	148	126	126
q7	2038	1618	1594	1594
q8	9930	2745	2724	2724
q9	10664	10447	10234	10234
q10	8674	3495	3525	3495
q11	430	246	247	246
q12	468	306	308	306
q13	18341	3956	4118	3956
q14	358	333	349	333
q15	510	469	462	462
q16	691	568	585	568
q17	1129	972	972	972
q18	7365	6857	7278	6857
q19	1793	1738	1573	1573
q20	554	352	317	317
q21	4467	4166	4179	4166
q22	564	469	485	469
Total cold run time: 117201 ms
Total hot run time: 49940 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4555	4588	4586	4586
q2	361	265	245	245
q3	4292	4255	4227	4227
q4	2772	2787	2778	2778
q5	7202	7173	7065	7065
q6	238	125	121	121
q7	3219	2810	2825	2810
q8	4356	4489	4502	4489
q9	16834	16733	16760	16733
q10	4245	4259	4271	4259
q11	775	713	674	674
q12	1028	864	875	864
q13	7332	3726	3784	3726
q14	453	433	428	428
q15	504	470	459	459
q16	744	693	692	692
q17	3779	3855	3891	3855
q18	8967	8858	8871	8858
q19	1717	1708	1648	1648
q20	2411	2183	2120	2120
q21	8493	8497	8539	8497
q22	1048	945	1002	945
Total cold run time: 85325 ms
Total hot run time: 80079 ms

doris-robot · 2024-07-01T07:49:28Z

TPC-DS: Total hot run time: 204699 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4321fae75262e0099fd96d3ff53c3209ff3d11d7, data reload: false

query1	925	423	378	378
query2	6527	2939	2660	2660
query3	6920	200	200	200
query4	20292	18043	17949	17949
query5	19757	6583	6641	6583
query6	294	218	245	218
query7	4169	303	307	303
query8	472	426	464	426
query9	3137	2691	2632	2632
query10	403	308	297	297
query11	11376	10648	10580	10580
query12	121	85	75	75
query13	5584	687	687	687
query14	17962	13654	13168	13168
query15	362	238	256	238
query16	6468	292	271	271
query17	1732	1468	887	887
query18	2291	414	411	411
query19	210	151	147	147
query20	82	79	79	79
query21	190	98	103	98
query22	5089	4942	4893	4893
query23	32496	31929	32027	31929
query24	7014	6560	6542	6542
query25	539	462	413	413
query26	527	162	168	162
query27	1844	305	300	300
query28	6071	2343	2304	2304
query29	2890	2769	2737	2737
query30	241	164	168	164
query31	899	730	740	730
query32	73	64	63	63
query33	405	276	249	249
query34	835	473	468	468
query35	1118	956	942	942
query36	1240	1147	1155	1147
query37	90	62	62	62
query38	3089	2914	2984	2914
query39	1349	1339	1345	1339
query40	204	102	99	99
query41	49	44	45	44
query42	85	84	86	84
query43	742	613	743	613
query44	1161	715	718	715
query45	251	243	239	239
query46	1228	967	961	961
query47	1818	1667	1879	1667
query48	1020	716	722	716
query49	618	380	389	380
query50	855	601	603	601
query51	4701	4609	4684	4609
query52	93	92	82	82
query53	463	334	323	323
query54	2667	2450	2482	2450
query55	87	78	76	76
query56	227	193	217	193
query57	1213	1087	1090	1087
query58	223	204	197	197
query59	4297	3974	4178	3974
query60	213	192	205	192
query61	99	92	95	92
query62	863	501	494	494
query63	486	344	354	344
query64	2505	1523	1511	1511
query65	3613	3584	3597	3584
query66	763	369	373	369
query67	16635	16669	17664	16669
query68	6840	662	651	651
query69	572	368	364	364
query70	1517	1390	1327	1327
query71	395	326	314	314
query72	6553	3526	3507	3507
query73	726	319	319	319
query74	6383	5859	5834	5834
query75	4759	3690	3724	3690
query76	4441	1122	1093	1093
query77	542	251	253	251
query78	12423	11595	11595	11595
query79	6923	661	663	661
query80	2557	398	405	398
query81	524	237	234	234
query82	1338	101	100	100
query83	184	134	138	134
query84	231	70	73	70
query85	1370	322	329	322
query86	364	310	291	291
query87	3277	3026	3046	3026
query88	4801	2327	2305	2305
query89	357	284	302	284
query90	1714	213	213	213
query91	176	144	139	139
query92	58	52	54	52
query93	2267	533	612	533
query94	885	217	212	212
query95	1116	1055	1079	1055
query96	638	327	328	327
query97	6415	6406	6357	6357
query98	188	177	174	174
query99	2947	889	857	857
Total cold run time: 305743 ms
Total hot run time: 204699 ms

doris-robot · 2024-07-01T07:54:53Z

ClickBench: Total hot run time: 30.71 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4321fae75262e0099fd96d3ff53c3209ff3d11d7, data reload: false

query1	0.03	0.03	0.02
query2	0.07	0.03	0.02
query3	0.26	0.03	0.05
query4	1.82	0.06	0.07
query5	0.54	0.53	0.52
query6	1.24	0.61	0.63
query7	0.01	0.01	0.01
query8	0.04	0.02	0.03
query9	0.53	0.49	0.48
query10	0.55	0.54	0.54
query11	0.12	0.09	0.10
query12	0.12	0.10	0.09
query13	0.62	0.62	0.60
query14	0.78	0.78	0.81
query15	0.78	0.76	0.77
query16	0.36	0.38	0.36
query17	1.02	0.97	1.00
query18	0.24	0.25	0.25
query19	1.87	1.84	1.88
query20	0.01	0.01	0.01
query21	15.48	0.55	0.57
query22	2.12	2.19	1.78
query23	16.86	0.96	1.07
query24	5.10	1.48	0.91
query25	0.34	0.12	0.05
query26	0.57	0.15	0.15
query27	0.04	0.04	0.04
query28	7.68	0.75	0.76
query29	12.68	2.13	2.22
query30	0.60	0.55	0.51
query31	2.80	0.39	0.38
query32	3.38	0.49	0.50
query33	3.06	3.06	3.06
query34	15.24	4.79	4.80
query35	4.87	4.85	4.84
query36	1.04	1.01	1.01
query37	0.06	0.05	0.04
query38	0.04	0.02	0.02
query39	0.02	0.01	0.01
query40	0.16	0.14	0.14
query41	0.06	0.01	0.02
query42	0.02	0.01	0.01
query43	0.03	0.02	0.02
Total cold run time: 103.26 s
Total hot run time: 30.71 s

doris-robot · 2024-07-01T07:58:08Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 4321fae75262e0099fd96d3ff53c3209ff3d11d7 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.6 seconds inserted 10000000 Rows, about 462K ops/s

…ctionCleanup Thread (apache#36942) pick (apache#36655) When using mysql catalog, mysql jdbc driver will generate an `AbandonedConnectionCleanupThread` for each database connection. This is a virtual reference, which will accumulate over time as database connections are constantly created, eventually causing OOM. Therefore, in our usage scenario, we need to turn off this thread because our database connection recycling depends on the connection pool. But please note that this switch is only for MySQL JDBC Driver versions greater than 8.0.22

morningman force-pushed the disable_mysql_clean_thread_20 branch from 3f2a70a to 4321fae Compare June 29, 2024 10:36

zy-kkk merged commit aedde47 into apache:branch-2.0 Jul 2, 2024

zy-kkk deleted the disable_mysql_clean_thread_20 branch July 2, 2024 08:36

[branch-2.0][improvement](mysql catalog) disable mysql AbandonedConnectionCleanup Thread #36942

[branch-2.0][improvement](mysql catalog) disable mysql AbandonedConnectionCleanup Thread #36942

Uh oh!

Conversation

zy-kkk commented Jun 27, 2024

Uh oh!

zy-kkk commented Jun 27, 2024

Uh oh!

doris-robot commented Jun 27, 2024

Uh oh!

doris-robot commented Jun 27, 2024

Uh oh!

doris-robot commented Jun 27, 2024

Uh oh!

doris-robot commented Jun 27, 2024

Uh oh!

doris-robot commented Jun 27, 2024

Uh oh!

morningman commented Jun 29, 2024

Uh oh!

doris-robot commented Jun 29, 2024

Uh oh!

doris-robot commented Jun 29, 2024

Uh oh!

doris-robot commented Jun 29, 2024

Uh oh!

doris-robot commented Jun 29, 2024

Uh oh!

zy-kkk commented Jul 1, 2024

Uh oh!

doris-robot commented Jul 1, 2024

Uh oh!

doris-robot commented Jul 1, 2024

Uh oh!

doris-robot commented Jul 1, 2024

Uh oh!

doris-robot commented Jul 1, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants