Skip to content

Conversation

@zy-kkk
Copy link
Member

@zy-kkk zy-kkk commented Aug 21, 2024

pick #39423

Force all resources to be closed in the close method. In the previous logic, query errors or query cancellation will not force the connection to be closed, which will cause abnormal Hikari connection counts. Although forced connection closure will generate some error logs in some cases, we should have this bottom-line guarantee and refine the closing logic later

@zy-kkk
Copy link
Member Author

zy-kkk commented Aug 21, 2024

run buildall

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zy-kkk
Copy link
Member Author

zy-kkk commented Aug 22, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 50616 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5d4b1561361e8e54586d0a9ac37250cf12cc2e1b, data reload: false

------ Round 1 ----------------------------------
q1	17850	4401	4389	4389
q2	2070	194	187	187
q3	10189	1885	1958	1885
q4	10375	1248	1341	1248
q5	8525	3962	3989	3962
q6	275	145	147	145
q7	2082	1642	1658	1642
q8	9324	2781	2750	2750
q9	11080	10597	10499	10499
q10	8694	3484	3537	3484
q11	448	280	280	280
q12	500	331	330	330
q13	18363	4012	4012	4012
q14	370	356	343	343
q15	556	502	508	502
q16	700	599	605	599
q17	1147	984	976	976
q18	7375	6883	6911	6883
q19	1753	1608	1613	1608
q20	591	320	352	320
q21	4455	4207	4122	4122
q22	523	450	452	450
Total cold run time: 117245 ms
Total hot run time: 50616 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4397	4345	4417	4345
q2	358	267	266	266
q3	4206	4133	4146	4133
q4	2784	2776	2759	2759
q5	7217	7170	7144	7144
q6	258	143	143	143
q7	3267	2877	2867	2867
q8	4383	4466	4500	4466
q9	17259	17087	17147	17087
q10	4242	4285	4329	4285
q11	793	683	699	683
q12	1044	859	862	859
q13	6610	3739	3740	3739
q14	478	455	443	443
q15	550	504	523	504
q16	767	733	715	715
q17	3833	3873	3861	3861
q18	8857	8811	8767	8767
q19	1783	1764	1691	1691
q20	2417	2149	2192	2149
q21	8538	8613	8617	8613
q22	1034	953	966	953
Total cold run time: 85075 ms
Total hot run time: 80472 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 209046 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5d4b1561361e8e54586d0a9ac37250cf12cc2e1b, data reload: false

query1	973	420	411	411
query2	6757	2282	2115	2115
query3	6946	230	224	224
query4	21492	18144	18152	18144
query5	19950	6721	6792	6721
query6	338	266	276	266
query7	4192	342	343	342
query8	465	468	474	468
query9	3266	2816	2772	2772
query10	527	369	374	369
query11	11427	10922	10772	10772
query12	161	118	108	108
query13	5671	727	725	725
query14	18238	13577	13647	13577
query15	387	252	246	246
query16	6515	330	303	303
query17	1605	1490	958	958
query18	2347	479	478	478
query19	251	182	180	180
query20	113	113	123	113
query21	241	153	145	145
query22	5276	5109	5067	5067
query23	32770	31951	31855	31855
query24	7043	6559	6554	6554
query25	588	495	468	468
query26	553	194	188	188
query27	1834	331	329	329
query28	6089	2440	2369	2369
query29	2936	2767	2846	2767
query30	289	213	206	206
query31	986	822	862	822
query32	104	97	92	92
query33	484	329	340	329
query34	895	520	519	519
query35	1208	1006	1041	1006
query36	1220	1325	997	997
query37	118	99	89	89
query38	3127	2949	2960	2949
query39	1512	1433	1447	1433
query40	266	150	149	149
query41	160	154	159	154
query42	110	112	110	110
query43	694	762	749	749
query44	1196	771	786	771
query45	282	264	272	264
query46	1278	1010	1053	1010
query47	2022	1769	1806	1769
query48	1047	740	733	733
query49	847	588	580	580
query50	959	734	683	683
query51	4829	4775	4722	4722
query52	117	110	104	104
query53	512	384	378	378
query54	2713	2511	2545	2511
query55	109	95	103	95
query56	322	305	277	277
query57	1240	1124	1119	1119
query58	319	312	304	304
query59	3614	3369	3426	3369
query60	292	295	293	293
query61	162	157	159	157
query62	855	590	577	577
query63	547	402	404	402
query64	2752	1722	1689	1689
query65	3702	3589	3603	3589
query66	1217	825	828	825
query67	15708	15253	15444	15253
query68	9094	694	699	694
query69	641	434	421	421
query70	1659	1497	1498	1497
query71	474	364	375	364
query72	6647	3501	3568	3501
query73	785	377	368	368
query74	6428	5909	5955	5909
query75	5540	3884	3835	3835
query76	5633	1187	1248	1187
query77	1101	416	423	416
query78	12804	11928	11847	11847
query79	6883	694	687	687
query80	1118	551	553	551
query81	536	290	287	287
query82	1480	133	125	125
query83	250	214	223	214
query84	290	102	103	102
query85	979	425	413	413
query86	361	330	326	326
query87	3251	3032	3030	3030
query88	5207	2573	2530	2530
query89	417	375	335	335
query90	2010	246	256	246
query91	194	163	163	163
query92	92	86	85	85
query93	4964	610	629	610
query94	738	256	251	251
query95	1169	1121	1120	1120
query96	646	345	342	342
query97	6507	6308	6442	6308
query98	220	210	207	207
query99	3020	963	1012	963
Total cold run time: 317903 ms
Total hot run time: 209046 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.15 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5d4b1561361e8e54586d0a9ac37250cf12cc2e1b, data reload: false

query1	0.02	0.03	0.02
query2	0.08	0.03	0.03
query3	0.26	0.06	0.06
query4	1.77	0.08	0.07
query5	0.55	0.52	0.52
query6	1.35	0.62	0.62
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.53	0.48	0.48
query10	0.54	0.54	0.55
query11	0.13	0.09	0.10
query12	0.15	0.10	0.10
query13	0.63	0.62	0.61
query14	0.79	0.80	0.80
query15	0.79	0.76	0.78
query16	0.37	0.37	0.39
query17	1.01	1.02	1.01
query18	0.22	0.26	0.22
query19	1.92	1.86	1.84
query20	0.02	0.01	0.01
query21	15.44	0.57	0.57
query22	2.12	2.57	1.61
query23	17.35	1.11	0.97
query24	6.29	0.84	0.92
query25	0.34	0.13	0.06
query26	0.68	0.15	0.16
query27	0.04	0.05	0.04
query28	7.08	0.73	0.74
query29	12.68	2.33	2.29
query30	0.76	0.68	0.71
query31	2.82	0.39	0.38
query32	3.35	0.51	0.51
query33	3.12	3.09	3.02
query34	15.27	4.84	4.83
query35	4.84	4.87	4.88
query36	1.06	1.02	1.02
query37	0.08	0.06	0.07
query38	0.05	0.04	0.03
query39	0.04	0.03	0.03
query40	0.18	0.17	0.18
query41	0.08	0.03	0.04
query42	0.04	0.04	0.03
query43	0.04	0.04	0.04
Total cold run time: 104.94 s
Total hot run time: 31.15 s

@zy-kkk zy-kkk merged commit f894a6b into apache:branch-2.0 Aug 22, 2024
@zy-kkk zy-kkk deleted the fix_jdbc_close_20 branch August 22, 2024 07:23
mongo360 pushed a commit to mongo360/doris that referenced this pull request Dec 11, 2024
…sed in the close method (apache#39665)

pick apache#39423

Force all resources to be closed in the close method. In the previous
logic, query errors or query cancellation will not force the connection
to be closed, which will cause abnormal Hikari connection counts.
Although forced connection closure will generate some error logs in some
cases, we should have this bottom-line guarantee and refine the closing
logic later
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants