Skip to content

[improvement](mysql catalog) disable mysql AbandonedConnectionCleanup Thread #36655

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
zy-kkk merged 1 commit into from
Jun 26, 2024
Merged

[improvement](mysql catalog) disable mysql AbandonedConnectionCleanup Thread #36655

zy-kkk merged 1 commit into from
Jun 26, 2024

Conversation

@zy-kkk
Copy link
Member

@zy-kkk zy-kkk commented Jun 21, 2024

When using mysql catalog, mysql jdbc driver will generate an AbandonedConnectionCleanupThread for each database connection. This is a virtual reference, which will accumulate over time as database connections are constantly created, eventually causing OOM. Therefore, in our usage scenario, we need to turn off this thread because our database connection recycling depends on the connection pool.
But please note that this switch is only for MySQL JDBC Driver versions greater than 8.0.22

@doris-robot
Copy link

doris-robot commented Jun 21, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zy-kkk
Copy link
Member Author

zy-kkk commented Jun 21, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Jun 21, 2024

TPC-H: Total hot run time: 39577 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ebb9f17613173979ef6fa2d90931c37ea42c254b, data reload: false

------ Round 1 ----------------------------------
q1	17619	4340	4205	4205
q2	2029	186	192	186
q3	10455	1136	1026	1026
q4	10198	806	801	801
q5	7460	2659	2679	2659
q6	222	142	147	142
q7	948	615	598	598
q8	9229	2076	2044	2044
q9	8895	6497	6474	6474
q10	8966	3669	3740	3669
q11	446	242	228	228
q12	447	234	221	221
q13	17773	2994	2963	2963
q14	271	230	223	223
q15	519	480	490	480
q16	544	375	382	375
q17	976	678	746	678
q18	7953	7438	7385	7385
q19	8314	1502	1386	1386
q20	667	324	338	324
q21	4882	3168	3907	3168
q22	389	342	349	342
Total cold run time: 119202 ms
Total hot run time: 39577 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4386	4240	4218	4218
q2	380	270	258	258
q3	3107	2882	2912	2882
q4	1970	1734	1770	1734
q5	5491	5517	5452	5452
q6	224	131	131	131
q7	2269	1824	1841	1824
q8	3257	3417	3408	3408
q9	8697	8744	8731	8731
q10	4161	3856	3800	3800
q11	600	499	517	499
q12	818	619	649	619
q13	17225	3175	3243	3175
q14	292	301	277	277
q15	514	492	498	492
q16	496	438	420	420
q17	1808	1528	1507	1507
q18	8083	8072	7861	7861
q19	1906	1655	1557	1557
q20	2195	1896	1869	1869
q21	10737	4775	4846	4775
q22	661	559	597	559
Total cold run time: 79277 ms
Total hot run time: 56048 ms

@doris-robot
Copy link

doris-robot commented Jun 21, 2024

TPC-DS: Total hot run time: 174164 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ebb9f17613173979ef6fa2d90931c37ea42c254b, data reload: false

query1	928	376	375	375
query2	6389	2381	2402	2381
query3	6635	206	217	206
query4	19121	17508	17318	17318
query5	3730	494	484	484
query6	245	168	160	160
query7	4591	314	287	287
query8	308	278	263	263
query9	8574	2462	2422	2422
query10	567	305	294	294
query11	10559	10043	10251	10043
query12	126	87	86	86
query13	1627	365	374	365
query14	10081	6921	6838	6838
query15	222	182	187	182
query16	7650	261	266	261
query17	1410	544	531	531
query18	1939	277	275	275
query19	197	164	161	161
query20	97	89	87	87
query21	207	128	129	128
query22	4305	4077	4033	4033
query23	33942	33715	33785	33715
query24	11328	2857	2930	2857
query25	609	392	395	392
query26	1157	159	172	159
query27	2313	323	316	316
query28	6102	2188	2149	2149
query29	902	643	621	621
query30	261	159	158	158
query31	960	768	782	768
query32	99	55	63	55
query33	765	286	297	286
query34	1091	481	477	477
query35	787	640	642	640
query36	1154	994	987	987
query37	147	77	76	76
query38	2982	2862	2829	2829
query39	890	851	831	831
query40	239	144	132	132
query41	55	52	55	52
query42	125	104	106	104
query43	604	564	536	536
query44	1191	739	741	739
query45	194	171	163	163
query46	1071	712	718	712
query47	1867	1781	1809	1781
query48	364	298	288	288
query49	842	415	412	412
query50	770	401	399	399
query51	6906	6843	6757	6757
query52	117	89	95	89
query53	362	300	287	287
query54	886	451	443	443
query55	75	75	73	73
query56	297	269	275	269
query57	1118	1090	1083	1083
query58	281	272	264	264
query59	3396	3084	3285	3084
query60	300	280	311	280
query61	96	90	92	90
query62	597	457	442	442
query63	313	290	299	290
query64	8693	2244	1761	1761
query65	3181	3091	3105	3091
query66	753	335	327	327
query67	15509	14987	14846	14846
query68	4550	533	551	533
query69	666	433	404	404
query70	1197	1073	1067	1067
query71	488	279	302	279
query72	7049	5493	5809	5493
query73	749	320	318	318
query74	5918	5483	5495	5483
query75	3638	2677	2696	2677
query76	2982	926	938	926
query77	686	311	307	307
query78	10446	9919	9698	9698
query79	2697	519	505	505
query80	2080	481	476	476
query81	559	222	224	222
query82	1392	118	106	106
query83	291	173	169	169
query84	257	95	85	85
query85	1455	287	275	275
query86	471	327	309	309
query87	3339	3090	3075	3075
query88	4023	2373	2347	2347
query89	481	389	403	389
query90	1762	196	193	193
query91	130	101	100	100
query92	59	52	54	52
query93	2557	498	500	498
query94	1086	192	189	189
query95	410	318	324	318
query96	600	276	265	265
query97	3207	3085	3081	3081
query98	225	202	270	202
query99	1292	833	855	833
Total cold run time: 271941 ms
Total hot run time: 174164 ms

@doris-robot
Copy link

doris-robot commented Jun 21, 2024

ClickBench: Total hot run time: 31.9 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ebb9f17613173979ef6fa2d90931c37ea42c254b, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.05
query3	0.22	0.05	0.05
query4	1.69	0.07	0.07
query5	0.50	0.47	0.51
query6	1.12	0.72	0.73
query7	0.02	0.02	0.01
query8	0.05	0.04	0.05
query9	0.55	0.49	0.49
query10	0.54	0.54	0.55
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.59	0.59	0.61
query14	0.77	0.78	0.79
query15	0.82	0.81	0.84
query16	0.35	0.38	0.37
query17	0.98	0.98	1.00
query18	0.23	0.23	0.25
query19	1.78	1.70	1.71
query20	0.01	0.01	0.01
query21	15.41	0.67	0.66
query22	3.91	6.27	3.22
query23	18.32	1.32	1.19
query24	2.12	0.24	0.23
query25	0.15	0.09	0.08
query26	0.27	0.18	0.17
query27	0.08	0.08	0.08
query28	13.27	1.04	1.00
query29	12.62	3.33	3.26
query30	0.25	0.06	0.06
query31	2.86	0.39	0.38
query32	3.28	0.47	0.46
query33	2.93	2.90	2.95
query34	16.98	4.42	4.42
query35	4.53	4.52	4.53
query36	0.66	0.48	0.48
query37	0.21	0.16	0.15
query38	0.16	0.15	0.16
query39	0.05	0.04	0.04
query40	0.18	0.15	0.15
query41	0.10	0.06	0.05
query42	0.05	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.09 s
Total hot run time: 31.9 s

morningman
morningman approved these changes Jun 21, 2024
View reviewed changes
Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 21, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Jun 21, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Jun 21, 2024

PR approved by anyone and no changes requested.

@zy-kkk zy-kkk merged commit ce0954f into apache:master Jun 26, 2024
@wm1581066 wm1581066 added the usercase Important user case type label label Jun 26, 2024
@zy-kkk zy-kkk deleted the disable_mysql_clean_thread branch June 27, 2024 11:07
zy-kkk added a commit to zy-kkk/doris that referenced this pull request Jun 27, 2024
@zy-kkk
[improvement](mysql catalog) disable mysql AbandonedConnectionCleanup…
3f2a70a 
… Thread (apache#36655)

When using mysql catalog, mysql jdbc driver will generate an
`AbandonedConnectionCleanupThread` for each database connection. This is
a virtual reference, which will accumulate over time as database
connections are constantly created, eventually causing OOM. Therefore,
in our usage scenario, we need to turn off this thread because our
database connection recycling depends on the connection pool.
But please note that this switch is only for MySQL JDBC Driver versions
greater than 8.0.22
@zy-kkk zy-kkk mentioned this pull request Jun 27, 2024
Merged
zy-kkk added a commit to zy-kkk/doris that referenced this pull request Jun 28, 2024
@zy-kkk
[improvement](mysql catalog) disable mysql AbandonedConnectionCleanup…
bfa57b5 
… Thread (apache#36655)

When using mysql catalog, mysql jdbc driver will generate an
`AbandonedConnectionCleanupThread` for each database connection. This is
a virtual reference, which will accumulate over time as database
connections are constantly created, eventually causing OOM. Therefore,
in our usage scenario, we need to turn off this thread because our
database connection recycling depends on the connection pool.
But please note that this switch is only for MySQL JDBC Driver versions
greater than 8.0.22
@zy-kkk zy-kkk mentioned this pull request Jun 28, 2024
Merged
dataroaring pushed a commit that referenced this pull request Jun 28, 2024
@zy-kkk @dataroaring
[improvement](mysql catalog) disable mysql AbandonedConnectionCleanup…
d958986 
… Thread (#36655)

When using mysql catalog, mysql jdbc driver will generate an
`AbandonedConnectionCleanupThread` for each database connection. This is
a virtual reference, which will accumulate over time as database
connections are constantly created, eventually causing OOM. Therefore,
in our usage scenario, we need to turn off this thread because our
database connection recycling depends on the connection pool.
But please note that this switch is only for MySQL JDBC Driver versions
greater than 8.0.22
morningman pushed a commit that referenced this pull request Jun 29, 2024
@zy-kkk
[branch-2.1][improvement](mysql catalog) disable mysql AbandonedConne…
e25b0d7 
…ctionCleanupThread (#36970)

pick (#36655)
morningman pushed a commit to zy-kkk/doris that referenced this pull request Jun 29, 2024
@zy-kkk @morningman
[improvement](mysql catalog) disable mysql AbandonedConnectionCleanup…
4321fae 
… Thread (apache#36655)

When using mysql catalog, mysql jdbc driver will generate an
`AbandonedConnectionCleanupThread` for each database connection. This is
a virtual reference, which will accumulate over time as database
connections are constantly created, eventually causing OOM. Therefore,
in our usage scenario, we need to turn off this thread because our
database connection recycling depends on the connection pool.
But please note that this switch is only for MySQL JDBC Driver versions
greater than 8.0.22
zy-kkk added a commit that referenced this pull request Jul 2, 2024
@zy-kkk
[branch-2.0][improvement](mysql catalog) disable mysql AbandonedConne…
aedde47 
…ctionCleanup Thread (#36942)

pick (#36655)

When using mysql catalog, mysql jdbc driver will generate an
`AbandonedConnectionCleanupThread` for each database connection. This is
a virtual reference, which will accumulate over time as database
connections are constantly created, eventually causing OOM. Therefore,
in our usage scenario, we need to turn off this thread because our
database connection recycling depends on the connection pool. But please
note that this switch is only for MySQL JDBC Driver versions greater
than 8.0.22
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@zy-kkk
[branch-2.0][improvement](mysql catalog) disable mysql AbandonedConne…
366e823 
…ctionCleanup Thread (apache#36942)

pick (apache#36655)

When using mysql catalog, mysql jdbc driver will generate an
`AbandonedConnectionCleanupThread` for each database connection. This is
a virtual reference, which will accumulate over time as database
connections are constantly created, eventually causing OOM. Therefore,
in our usage scenario, we need to turn off this thread because our
database connection recycling depends on the connection pool. But please
note that this switch is only for MySQL JDBC Driver versions greater
than 8.0.22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morningman morningman morningman approved these changes

@AshinGau AshinGau AshinGau approved these changes

@wuwenchi wuwenchi wuwenchi approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.0.13-merged dev/2.1.5-merged dev/3.0.0-merged reviewed usercase Important user case type label

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@zy-kkk @doris-robot @morningman @AshinGau @wuwenchi @wm1581066 @dataroaring