Skip to content

Conversation

@zy-kkk
Copy link
Member

@zy-kkk zy-kkk commented Jun 28, 2024

pick (#36720)

In many cases, we found that users would use JDBC Catalog to perform a large number of queries, which resulted in the maximum of 10 connections being insufficient, so I adjusted it to 30, which covered most needs.

…in the connection pool to 30 by default (apache#36720)

In many cases, we found that users would use JDBC Catalog to perform a
large number of queries, which resulted in the maximum of 10 connections
being insufficient, so I adjusted it to 30, which covered most needs.
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zy-kkk
Copy link
Member Author

zy-kkk commented Jun 28, 2024

run buildall

@zy-kkk zy-kkk changed the title [improvement](jdbc catalog) Modify the maximum number of connections in the connection pool to 30 by default [branch-2.0][improvement](jdbc catalog) Modify the maximum number of connections in the connection pool to 30 by default Jun 28, 2024
@doris-robot
Copy link

TPC-H: Total hot run time: 49779 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c2c934f6d3d4cbbed543081ca7f3c70428da6a44, data reload: false

------ Round 1 ----------------------------------
q1	18050	4545	4325	4325
q2	2071	158	145	145
q3	10483	1894	1968	1894
q4	10347	1289	1342	1289
q5	8413	3878	3987	3878
q6	239	157	131	131
q7	2095	1641	1580	1580
q8	9552	2754	2717	2717
q9	13836	10317	10308	10308
q10	8630	3543	3542	3542
q11	415	249	246	246
q12	468	310	309	309
q13	18339	3957	4052	3957
q14	362	323	323	323
q15	523	473	465	465
q16	663	570	573	570
q17	1140	935	892	892
q18	7264	6798	6958	6798
q19	1804	1694	1607	1607
q20	519	317	307	307
q21	4454	4132	4057	4057
q22	530	456	439	439
Total cold run time: 120197 ms
Total hot run time: 49779 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4326	4368	4326	4326
q2	329	215	214	214
q3	4194	4150	4145	4145
q4	2754	2755	2754	2754
q5	7169	7075	7048	7048
q6	245	122	127	122
q7	3237	2820	2925	2820
q8	4354	4432	4512	4432
q9	16987	16774	16683	16683
q10	4261	4307	4305	4305
q11	767	690	706	690
q12	1032	875	859	859
q13	7127	3746	3756	3746
q14	452	423	419	419
q15	508	464	458	458
q16	736	683	686	683
q17	3857	3810	3884	3810
q18	8796	8725	8766	8725
q19	1745	1665	1672	1665
q20	2357	2113	2158	2113
q21	8452	8475	8601	8475
q22	1044	963	1005	963
Total cold run time: 84729 ms
Total hot run time: 79455 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 205576 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c2c934f6d3d4cbbed543081ca7f3c70428da6a44, data reload: false

query1	940	430	382	382
query2	6541	2934	2627	2627
query3	6921	206	210	206
query4	20582	18008	17985	17985
query5	19726	6477	6578	6477
query6	301	225	236	225
query7	4164	309	312	309
query8	441	423	424	423
query9	3144	2684	2635	2635
query10	402	324	293	293
query11	11388	10696	10785	10696
query12	129	78	75	75
query13	5602	699	693	693
query14	17718	13601	13954	13601
query15	361	246	239	239
query16	6471	308	268	268
query17	1719	1446	889	889
query18	2309	413	414	413
query19	215	149	152	149
query20	80	78	83	78
query21	195	97	99	97
query22	5261	5107	5131	5107
query23	32740	31970	32019	31970
query24	7071	6530	6510	6510
query25	556	450	440	440
query26	609	167	164	164
query27	1993	294	301	294
query28	6113	2355	2303	2303
query29	2842	2811	2742	2742
query30	247	168	166	166
query31	887	722	746	722
query32	68	63	60	60
query33	400	263	270	263
query34	839	476	487	476
query35	1116	929	933	929
query36	1307	1306	1113	1113
query37	88	62	60	60
query38	3055	2910	2902	2902
query39	1378	1336	1340	1336
query40	199	97	96	96
query41	50	43	44	43
query42	82	82	89	82
query43	726	819	682	682
query44	1139	707	716	707
query45	248	230	237	230
query46	1219	955	955	955
query47	1873	1801	1774	1774
query48	1027	722	708	708
query49	615	374	379	374
query50	868	622	602	602
query51	4738	4631	4627	4627
query52	103	81	87	81
query53	454	334	335	334
query54	2670	2427	2500	2427
query55	85	80	77	77
query56	246	223	218	218
query57	1229	1119	1206	1119
query58	228	190	209	190
query59	4609	4337	3956	3956
query60	222	218	200	200
query61	98	93	95	93
query62	792	484	492	484
query63	497	345	346	345
query64	2591	1528	1401	1401
query65	3658	3576	3579	3576
query66	813	399	387	387
query67	15460	15653	15516	15516
query68	8673	676	645	645
query69	571	339	352	339
query70	1527	1491	1451	1451
query71	405	301	307	301
query72	6479	3509	3452	3452
query73	727	323	320	320
query74	6272	5820	5850	5820
query75	5361	3771	3710	3710
query76	5413	1193	1197	1193
query77	853	250	266	250
query78	12621	12824	13638	12824
query79	12246	650	638	638
query80	916	396	397	396
query81	500	234	233	233
query82	637	99	100	99
query83	181	135	134	134
query84	259	75	69	69
query85	752	317	313	313
query86	337	308	307	307
query87	3237	3061	2999	2999
query88	6015	2328	2338	2328
query89	378	290	278	278
query90	2595	208	218	208
query91	171	140	148	140
query92	58	52	51	51
query93	4442	553	607	553
query94	1187	208	208	208
query95	1090	1060	1072	1060
query96	642	321	332	321
query97	6478	6373	6374	6373
query98	198	178	171	171
query99	2709	791	871	791
Total cold run time: 316388 ms
Total hot run time: 205576 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.51 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c2c934f6d3d4cbbed543081ca7f3c70428da6a44, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.03	0.03
query3	0.25	0.05	0.04
query4	1.79	0.07	0.07
query5	0.54	0.52	0.51
query6	1.24	0.62	0.61
query7	0.01	0.01	0.01
query8	0.03	0.03	0.02
query9	0.52	0.49	0.47
query10	0.57	0.55	0.53
query11	0.12	0.09	0.08
query12	0.12	0.10	0.09
query13	0.62	0.62	0.61
query14	0.80	0.78	0.80
query15	0.78	0.76	0.75
query16	0.36	0.38	0.36
query17	1.00	1.00	1.03
query18	0.24	0.25	0.25
query19	1.94	1.83	1.90
query20	0.02	0.01	0.01
query21	15.46	0.56	0.55
query22	2.16	2.18	1.44
query23	17.03	1.08	0.95
query24	7.13	0.87	0.98
query25	0.37	0.07	0.05
query26	0.78	0.16	0.15
query27	0.04	0.05	0.04
query28	6.25	0.73	0.77
query29	12.62	2.32	2.30
query30	0.61	0.53	0.56
query31	2.81	0.39	0.38
query32	3.38	0.49	0.49
query33	3.05	3.10	3.08
query34	15.26	4.81	4.78
query35	4.87	4.84	4.86
query36	1.04	1.01	1.01
query37	0.06	0.04	0.05
query38	0.03	0.02	0.02
query39	0.02	0.01	0.01
query40	0.16	0.14	0.14
query41	0.06	0.01	0.01
query42	0.02	0.01	0.01
query43	0.02	0.02	0.02
Total cold run time: 104.27 s
Total hot run time: 30.51 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit c2c934f6d3d4cbbed543081ca7f3c70428da6a44 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.3 seconds inserted 10000000 Rows, about 469K ops/s

@morningman morningman merged commit 8802608 into apache:branch-2.0 Jul 1, 2024
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
…connections in the connection pool to 30 by default (apache#37024)

pick (apache#36720)

In many cases, we found that users would use JDBC Catalog to perform a
large number of queries, which resulted in the maximum of 10 connections
being insufficient, so I adjusted it to 30, which covered most needs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants