[branch-2.0][fix](jdbc catalog) Fix Memory Leak by Enabling Weak References in HikariCP

[zy-kkk]

pick (#39582)

This PR addresses a memory leak issue caused by FastList objects in HikariCP being retained by ThreadLocal variables, which are not easily garbage collected in long-running JNI threads. To mitigate this, a system property com.zaxxer.hikari.useWeakReferences is set to true, ensuring that WeakReference is used for ThreadLocal objects, allowing the garbage collector to reclaim memory more effectively. Even though setting this will affect some performance, solving resource leaks is relatively more important
Performance difference before and after setting
Before setting:
10 concurrency 0.02-0.05
100 concurrency 0.18-0.4
After setting:
10 concurrency 0.02-0.07
100 concurrency 0.18-0.7

[doris-robot]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

[zy-kkk]

run buildall

[doris-robot]

TPC-H: Total hot run time: 49825 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 446d9967abd0c23cc28c432237bc70d193b6ddac, data reload: false

------ Round 1 ----------------------------------
q1	18083	4376	4363	4363
q2	2028	181	146	146
q3	10232	1947	1932	1932
q4	10349	1267	1318	1267
q5	8559	3856	3848	3848
q6	235	123	124	123
q7	2031	1598	1617	1598
q8	9277	2734	2699	2699
q9	10394	10243	10270	10243
q10	8623	3571	3532	3532
q11	418	252	238	238
q12	460	302	298	298
q13	18378	4071	4196	4071
q14	353	324	325	324
q15	499	461	463	461
q16	665	572	579	572
q17	1158	966	962	962
q18	7178	6818	6811	6811
q19	1685	1587	1567	1567
q20	572	308	298	298
q21	4446	4079	4122	4079
q22	493	393	397	393
Total cold run time: 116116 ms
Total hot run time: 49825 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4328	4350	4366	4350
q2	320	228	222	222
q3	4137	4170	4147	4147
q4	2759	2732	2755	2732
q5	7192	7112	7090	7090
q6	239	116	123	116
q7	3234	2800	2801	2800
q8	4388	4496	4506	4496
q9	17044	16992	16873	16873
q10	4306	4291	4312	4291
q11	763	684	701	684
q12	1047	837	874	837
q13	6935	3762	3762	3762
q14	454	432	427	427
q15	494	465	474	465
q16	756	681	677	677
q17	3802	3906	3836	3836
q18	8926	8897	9225	8897
q19	1726	1763	1630	1630
q20	2494	2254	2212	2212
q21	8848	8866	8524	8524
q22	1015	984	921	921
Total cold run time: 85207 ms
Total hot run time: 79989 ms

[doris-robot]

TPC-DS: Total hot run time: 202943 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 446d9967abd0c23cc28c432237bc70d193b6ddac, data reload: false

query1	920	392	432	392
query2	6554	2227	2239	2227
query3	6919	206	200	200
query4	20797	18264	18111	18111
query5	19733	6527	6534	6527
query6	405	214	236	214
query7	5009	315	309	309
query8	283	240	269	240
query9	3105	2703	2636	2636
query10	404	297	301	297
query11	11343	10628	10869	10628
query12	124	84	72	72
query13	5607	662	647	647
query14	19428	13387	13494	13387
query15	357	221	225	221
query16	6430	300	269	269
query17	1565	1451	878	878
query18	2301	430	421	421
query19	221	148	156	148
query20	80	81	78	78
query21	188	99	94	94
query22	5209	5196	5331	5196
query23	32552	31987	31881	31881
query24	6776	6508	6541	6508
query25	539	426	424	424
query26	518	171	164	164
query27	1732	301	299	299
query28	6097	2314	2270	2270
query29	2923	2753	2686	2686
query30	255	167	167	167
query31	951	778	785	778
query32	68	64	57	57
query33	403	258	257	257
query34	850	474	481	474
query35	1124	919	933	919
query36	1388	1344	1134	1134
query37	92	60	63	60
query38	3090	2900	2943	2900
query39	1377	1297	1337	1297
query40	207	101	98	98
query41	40	38	38	38
query42	88	80	91	80
query43	607	640	603	603
query44	1162	721	713	713
query45	241	228	231	228
query46	1246	964	980	964
query47	1803	1728	1724	1724
query48	998	706	697	697
query49	622	368	366	366
query50	879	625	628	625
query51	4757	4652	4707	4652
query52	99	88	86	86
query53	443	321	325	321
query54	2704	2485	2494	2485
query55	88	75	78	75
query56	234	212	205	205
query57	1137	1144	1133	1133
query58	222	200	185	185
query59	3656	3215	3177	3177
query60	226	195	200	195
query61	99	98	97	97
query62	844	475	494	475
query63	497	341	338	338
query64	2512	1497	1436	1436
query65	3656	3563	3591	3563
query66	752	381	386	381
query67	16227	15694	15936	15694
query68	8503	683	632	632
query69	581	343	349	343
query70	1607	1675	1490	1490
query71	419	316	324	316
query72	6526	3445	3525	3445
query73	724	321	324	321
query74	6286	5895	5816	5816
query75	5201	3741	3760	3741
query76	5117	1137	1218	1137
query77	751	260	250	250
query78	12490	11891	11631	11631
query79	8370	629	633	629
query80	1470	395	395	395
query81	490	233	232	232
query82	1556	101	98	98
query83	178	134	134	134
query84	262	73	72	72
query85	885	315	314	314
query86	330	326	295	295
query87	3252	2982	3062	2982
query88	4801	2284	2285	2284
query89	365	307	294	294
query90	2069	205	201	201
query91	154	122	121	121
query92	61	48	54	48
query93	6219	542	595	542
query94	738	212	206	206
query95	1122	1077	1064	1064
query96	650	325	323	323
query97	6492	6388	6394	6388
query98	183	177	166	166
query99	3071	932	1013	932
Total cold run time: 314686 ms
Total hot run time: 202943 ms

[doris-robot]

ClickBench: Total hot run time: 30.88 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 446d9967abd0c23cc28c432237bc70d193b6ddac, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.02	0.02
query3	0.25	0.05	0.05
query4	1.78	0.06	0.06
query5	0.54	0.52	0.52
query6	1.26	0.62	0.62
query7	0.01	0.00	0.01
query8	0.03	0.02	0.03
query9	0.54	0.50	0.47
query10	0.54	0.54	0.56
query11	0.12	0.08	0.09
query12	0.12	0.09	0.09
query13	0.62	0.61	0.61
query14	0.78	0.79	0.79
query15	0.78	0.77	0.76
query16	0.37	0.37	0.40
query17	0.99	1.03	1.02
query18	0.22	0.28	0.21
query19	1.94	1.85	1.84
query20	0.01	0.01	0.01
query21	15.46	0.60	0.55
query22	2.14	2.36	1.58
query23	17.25	1.14	0.96
query24	4.36	1.20	1.08
query25	0.31	0.15	0.04
query26	0.56	0.16	0.16
query27	0.04	0.05	0.04
query28	8.44	0.76	0.73
query29	12.65	2.30	2.38
query30	0.60	0.53	0.53
query31	2.82	0.39	0.38
query32	3.35	0.51	0.50
query33	3.22	3.04	3.05
query34	15.27	4.80	4.78
query35	4.90	4.84	4.85
query36	1.06	1.01	1.02
query37	0.06	0.05	0.05
query38	0.04	0.02	0.02
query39	0.02	0.01	0.01
query40	0.16	0.14	0.15
query41	0.06	0.01	0.01
query42	0.02	0.01	0.01
query43	0.03	0.02	0.02
Total cold run time: 103.81 s
Total hot run time: 30.88 s

[doris-robot]

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 446d9967abd0c23cc28c432237bc70d193b6ddac with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.5 seconds inserted 10000000 Rows, about 465K ops/s