Skip to content

[fix](third-party) enable keepalive on socket created by libevent #35805

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
Jun 4, 2024
Merged

[fix](third-party) enable keepalive on socket created by libevent #35805

dataroaring merged 1 commit into from
Jun 4, 2024

Conversation

@dataroaring
Copy link
Contributor

@dataroaring dataroaring commented Jun 3, 2024 •
edited
Loading

Proposed changes

We encounter a problem that tcp connections created by libevent is in established while there is no corresponding endpoint in the other endpoint. So we enable keepalive to recycle the tcp connection.

@doris-robot
Copy link

doris-robot commented Jun 3, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@dataroaring
Copy link
Contributor Author

dataroaring commented Jun 3, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Jun 3, 2024

TPC-H: Total hot run time: 41048 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 84da644d3a4f83e239ef5386e69656a55bc816c7, data reload: false

------ Round 1 ----------------------------------
q1	17875	4518	4441	4441
q2	2559	208	221	208
q3	10871	1336	1195	1195
q4	11119	866	833	833
q5	8125	2746	2706	2706
q6	231	142	148	142
q7	988	631	612	612
q8	9255	2098	2127	2098
q9	9080	6450	6481	6450
q10	8974	3777	3757	3757
q11	466	248	247	247
q12	433	240	240	240
q13	18832	2998	3023	2998
q14	268	228	219	219
q15	515	492	463	463
q16	523	387	381	381
q17	985	723	714	714
q18	7980	7332	7370	7332
q19	7962	1461	1464	1461
q20	672	316	312	312
q21	4883	4010	3906	3906
q22	392	340	333	333
Total cold run time: 122988 ms
Total hot run time: 41048 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4395	4240	4258	4240
q2	368	265	261	261
q3	3005	2775	2789	2775
q4	1865	1647	1638	1638
q5	5252	5259	5240	5240
q6	221	130	131	130
q7	2149	1713	1731	1713
q8	3187	3301	3322	3301
q9	8379	8311	8310	8310
q10	3926	3674	3687	3674
q11	580	492	489	489
q12	760	588	620	588
q13	16765	2994	3016	2994
q14	298	276	271	271
q15	513	472	476	472
q16	481	414	416	414
q17	1766	1491	1475	1475
q18	7542	7464	7440	7440
q19	1777	1623	1673	1623
q20	1989	1792	1791	1791
q21	4962	4821	4708	4708
q22	645	542	527	527
Total cold run time: 70825 ms
Total hot run time: 54074 ms

@doris-robot
Copy link

doris-robot commented Jun 3, 2024

TPC-DS: Total hot run time: 170444 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 84da644d3a4f83e239ef5386e69656a55bc816c7, data reload: false

query1	926	376	371	371
query2	6444	2404	2370	2370
query3	6651	210	209	209
query4	20345	17283	17399	17283
query5	4140	465	472	465
query6	259	162	157	157
query7	4595	305	299	299
query8	341	295	279	279
query9	8553	2439	2408	2408
query10	463	310	298	298
query11	10664	10034	10134	10034
query12	150	92	95	92
query13	1652	378	372	372
query14	10290	7708	7337	7337
query15	252	186	186	186
query16	7920	290	277	277
query17	1811	536	541	536
query18	1945	279	268	268
query19	208	167	163	163
query20	95	91	88	88
query21	220	126	130	126
query22	4676	4149	4044	4044
query23	33633	33013	33223	33013
query24	11859	2821	2840	2821
query25	670	357	364	357
query26	1756	158	158	158
query27	3008	323	330	323
query28	7434	2105	2073	2073
query29	1026	605	601	601
query30	294	148	155	148
query31	963	733	723	723
query32	95	55	57	55
query33	764	294	291	291
query34	959	490	478	478
query35	770	611	617	611
query36	1073	924	925	924
query37	195	71	73	71
query38	2875	2739	2743	2739
query39	890	795	802	795
query40	283	129	126	126
query41	53	52	53	52
query42	119	96	98	96
query43	580	539	538	538
query44	1197	728	757	728
query45	200	165	176	165
query46	1091	748	717	717
query47	1856	1740	1744	1740
query48	380	293	289	289
query49	1178	422	419	419
query50	794	408	399	399
query51	6919	6739	6748	6739
query52	104	100	94	94
query53	360	293	296	293
query54	927	457	453	453
query55	74	73	74	73
query56	285	268	262	262
query57	1125	1024	1031	1024
query58	280	255	258	255
query59	3491	3019	3341	3019
query60	333	280	269	269
query61	91	92	89	89
query62	648	435	445	435
query63	326	305	287	287
query64	9952	2255	1711	1711
query65	3177	3068	3115	3068
query66	1349	334	336	334
query67	15426	15052	15132	15052
query68	5374	562	551	551
query69	617	454	394	394
query70	1215	1120	1115	1115
query71	441	295	292	292
query72	7449	2706	2521	2521
query73	823	327	319	319
query74	5963	5503	5524	5503
query75	3787	2669	2644	2644
query76	3252	871	935	871
query77	660	312	291	291
query78	10551	9851	9715	9715
query79	1980	528	518	518
query80	1103	492	475	475
query81	566	218	220	218
query82	759	106	102	102
query83	209	172	170	170
query84	271	89	90	89
query85	1412	293	286	286
query86	467	295	299	295
query87	3257	3088	3104	3088
query88	3866	2357	2371	2357
query89	489	394	395	394
query90	1813	198	198	198
query91	137	113	178	113
query92	58	49	52	49
query93	1857	520	506	506
query94	1193	203	186	186
query95	417	317	317	317
query96	588	268	274	268
query97	3178	2991	3043	2991
query98	251	217	220	217
query99	1376	822	851	822
Total cold run time: 278841 ms
Total hot run time: 170444 ms

@doris-robot
Copy link

doris-robot commented Jun 3, 2024

TeamCity be ut coverage result:
Function Coverage: 36.29% (9240/25460)
Line Coverage: 27.66% (75862/274277)
Region Coverage: 26.89% (39310/146195)
Branch Coverage: 23.61% (19918/84362)
Coverage Report: http://coverage.selectdb-in.cc/coverage/84da644d3a4f83e239ef5386e69656a55bc816c7_84da644d3a4f83e239ef5386e69656a55bc816c7/report/index.html

@doris-robot
Copy link

doris-robot commented Jun 3, 2024

ClickBench: Total hot run time: 30.72 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 84da644d3a4f83e239ef5386e69656a55bc816c7, data reload: false

query1	0.04	0.03	0.04
query2	0.09	0.04	0.04
query3	0.23	0.06	0.05
query4	1.64	0.09	0.09
query5	0.51	0.47	0.51
query6	1.14	0.73	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.54	0.48	0.49
query10	0.53	0.55	0.55
query11	0.16	0.11	0.12
query12	0.15	0.13	0.12
query13	0.59	0.58	0.60
query14	0.77	0.77	0.79
query15	0.84	0.82	0.80
query16	0.36	0.37	0.38
query17	0.97	1.04	1.04
query18	0.22	0.28	0.25
query19	1.92	1.75	1.72
query20	0.02	0.01	0.02
query21	15.70	0.66	0.65
query22	4.13	6.98	2.06
query23	18.26	1.41	1.28
query24	2.11	0.23	0.20
query25	0.15	0.09	0.08
query26	0.27	0.18	0.18
query27	0.08	0.08	0.08
query28	13.24	1.03	1.02
query29	13.16	3.34	3.29
query30	0.24	0.06	0.05
query31	2.88	0.39	0.37
query32	3.25	0.50	0.46
query33	2.87	2.93	2.86
query34	16.86	4.38	4.37
query35	4.47	4.43	4.47
query36	0.66	0.48	0.47
query37	0.18	0.16	0.16
query38	0.16	0.15	0.15
query39	0.05	0.04	0.04
query40	0.17	0.14	0.15
query41	0.09	0.04	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.87 s
Total hot run time: 30.72 s

@github-actions
Copy link
Contributor

github-actions bot commented Jun 3, 2024

PR approved by anyone and no changes requested.

morningman
morningman approved these changes Jun 4, 2024
View reviewed changes
Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 4, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Jun 4, 2024

PR approved by at least one committer and no changes requested.

@dataroaring dataroaring merged commit afc0078 into apache:master Jun 4, 2024
dataroaring added a commit that referenced this pull request Jun 4, 2024
@dataroaring
[fix](third-party) enable keepalive on socket created by libevent (#3…
b9947e2 
…5805)


We encounter a problem that tcp connections created by libevent is in
established while there is no corresponding endpoint in the other
endpoint. So we enable keepalive to recycle the tcp connection.
seawinde pushed a commit to seawinde/doris that referenced this pull request Jun 5, 2024
@dataroaring @seawinde
[fix](third-party) enable keepalive on socket created by libevent (ap…
9d19fe6 
…ache#35805)


We encounter a problem that tcp connections created by libevent is in
established while there is no corresponding endpoint in the other
endpoint. So we enable keepalive to recycle the tcp connection.
@morningman morningman mentioned this pull request Jun 8, 2024
Merged
dataroaring added a commit that referenced this pull request Jun 10, 2024
@dataroaring
[fix](thirdparty) patch libevent to enable keepalive on tcp (#36026)
9e852a7 
Followup #35805, fix patch issue and compile issue of libevent
dataroaring added a commit to dataroaring/incubator-doris that referenced this pull request Jun 11, 2024
@dataroaring
[fix](third-party) enable keepalive on socket created by libevent (ap…
4f4d694 
…ache#35805)


We encounter a problem that tcp connections created by libevent is in
established while there is no corresponding endpoint in the other
endpoint. So we enable keepalive to recycle the tcp connection.
dataroaring added a commit to dataroaring/incubator-doris that referenced this pull request Jun 11, 2024
@dataroaring
[fix](thirdparty) patch libevent to enable keepalive on tcp (apache#3…
29241bc 
…6026)

Followup apache#35805, fix patch issue and compile issue of libevent
@dataroaring dataroaring mentioned this pull request Jun 11, 2024
Merged
dataroaring added a commit to dataroaring/incubator-doris that referenced this pull request Jun 11, 2024
@dataroaring
[fix](third-party) enable keepalive on socket created by libevent (ap…
eb2df00 
…ache#35805)


We encounter a problem that tcp connections created by libevent is in
established while there is no corresponding endpoint in the other
endpoint. So we enable keepalive to recycle the tcp connection.
dataroaring added a commit to dataroaring/incubator-doris that referenced this pull request Jun 11, 2024
@dataroaring
[fix](thirdparty) patch libevent to enable keepalive on tcp (apache#3…
93e84a5 
…6026)

Followup apache#35805, fix patch issue and compile issue of libevent
@dataroaring dataroaring mentioned this pull request Jun 11, 2024
Merged
dataroaring added a commit that referenced this pull request Jun 11, 2024
@dataroaring
[fix](thirdparty) enable keepalive on socket created by libevent (#36092
b3e94a4 
)

## Proposed changes

pick #35805
#36026

Issue Number: close #xxx

<!--Describe your changes.-->
dataroaring added a commit to dataroaring/incubator-doris that referenced this pull request Jun 11, 2024
@dataroaring
[fix](third-party) enable keepalive on socket created by libevent (ap…
1bb706a 
…ache#35805)


We encounter a problem that tcp connections created by libevent is in
established while there is no corresponding endpoint in the other
endpoint. So we enable keepalive to recycle the tcp connection.
dataroaring added a commit to dataroaring/incubator-doris that referenced this pull request Jun 11, 2024
@dataroaring
[fix](thirdparty) patch libevent to enable keepalive on tcp (apache#3…
e23a12c 
…6026)

Followup apache#35805, fix patch issue and compile issue of libevent
dataroaring added a commit that referenced this pull request Jun 11, 2024
@dataroaring
[fix](third-party) enable keepalive on socket created by libevent (#3…
fce09ae 
…6088)

pick #35805 #36026
dataroaring added a commit that referenced this pull request Jun 11, 2024
@dataroaring
[fix](third-party) enable keepalive on socket created by libevent (#3…
c58146e 
…6097)

pick #35805 #36026
@dataroaring dataroaring mentioned this pull request Jun 11, 2024
Merged
dataroaring added a commit that referenced this pull request Jun 13, 2024
@dataroaring
[fix](thirdparty) patch libevent to enable keepalive on tcp (#36026)
e9b19f3 
Followup #35805, fix patch issue and compile issue of libevent
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@dataroaring
[fix](third-party) enable keepalive on socket created by libevent (ap…
f8fa985 
…ache#36097)

pick apache#35805 apache#36026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morningman morningman morningman approved these changes

@gavinchou gavinchou gavinchou approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.0.12-merged dev/2.1.4-merged dev/3.0.0-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@dataroaring @doris-robot @morningman @gavinchou