Skip to content

[bugfix](handshake) brpc handshake should not use light pool #42115

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
yiguolei merged 2 commits into from
Oct 19, 2024
Merged

[bugfix](handshake) brpc handshake should not use light pool #42115

yiguolei merged 2 commits into from
Oct 19, 2024

Conversation

@yiguolei
Copy link
Contributor

@yiguolei yiguolei commented Oct 18, 2024

Proposed changes

The light pool may be full. Handshake is used to check the connection state of brpc. Should not be interfered by the thread pool logic.

@doris-robot
Copy link

doris-robot commented Oct 18, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@github-actions
Copy link
Contributor

github-actions bot commented Oct 18, 2024

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions
Copy link
Contributor

github-actions bot commented Oct 18, 2024

clang-tidy review says "All clean, LGTM! 👍"

mrhhsg
mrhhsg approved these changes Oct 18, 2024
View reviewed changes
Copy link
Member

@mrhhsg mrhhsg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@github-actions
Copy link
Contributor

github-actions bot commented Oct 18, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Oct 18, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Oct 18, 2024

PR approved by anyone and no changes requested.

@yiguolei
Copy link
Contributor Author

yiguolei commented Oct 18, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Oct 18, 2024

TPC-H: Total hot run time: 41111 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 10498e8d641e5e924121ba4c247b6c8adc92e60c, data reload: false

------ Round 1 ----------------------------------
q1	17571	7491	7353	7353
q2	2065	154	154	154
q3	10577	1134	1162	1134
q4	10229	816	837	816
q5	7733	3111	3077	3077
q6	234	155	148	148
q7	1010	608	598	598
q8	9430	1921	1971	1921
q9	7178	6438	6419	6419
q10	7111	2437	2415	2415
q11	459	248	246	246
q12	515	213	217	213
q13	17795	2986	2984	2984
q14	232	207	209	207
q15	572	527	515	515
q16	673	599	570	570
q17	969	548	498	498
q18	7438	6658	6850	6658
q19	1357	1029	945	945
q20	474	179	180	179
q21	3936	3228	3053	3053
q22	1117	1008	1012	1008
Total cold run time: 108675 ms
Total hot run time: 41111 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7273	7220	7201	7201
q2	330	227	221	221
q3	2944	2801	2783	2783
q4	1956	1719	1715	1715
q5	5466	5484	5502	5484
q6	233	141	142	141
q7	2153	1742	1716	1716
q8	3254	3419	3404	3404
q9	8575	8580	8576	8576
q10	3532	3495	3447	3447
q11	587	474	483	474
q12	804	617	566	566
q13	11743	2970	3023	2970
q14	296	257	272	257
q15	566	515	520	515
q16	688	639	627	627
q17	1831	1593	1557	1557
q18	7808	7435	7566	7435
q19	1677	1571	1587	1571
q20	2035	1823	1838	1823
q21	5311	5282	5210	5210
q22	1114	1013	1004	1004
Total cold run time: 70176 ms
Total hot run time: 58697 ms

@doris-robot
Copy link

doris-robot commented Oct 18, 2024

TeamCity be ut coverage result:
Function Coverage: 37.47% (9710/25915)
Line Coverage: 28.75% (80652/280533)
Region Coverage: 28.18% (41723/148045)
Branch Coverage: 24.76% (21211/85658)
Coverage Report: http://coverage.selectdb-in.cc/coverage/10498e8d641e5e924121ba4c247b6c8adc92e60c_10498e8d641e5e924121ba4c247b6c8adc92e60c/report/index.html

@doris-robot
Copy link

doris-robot commented Oct 18, 2024

TPC-DS: Total hot run time: 192758 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 10498e8d641e5e924121ba4c247b6c8adc92e60c, data reload: false

query1	952	405	403	403
query2	3609	2174	2041	2041
query3	3048	213	231	213
query4	32957	23898	23693	23693
query5	3074	480	463	463
query6	242	167	185	167
query7	3453	295	289	289
query8	283	235	221	221
query9	7912	2735	2740	2735
query10	438	283	289	283
query11	17916	15471	15325	15325
query12	151	103	102	102
query13	1388	437	443	437
query14	9262	7554	7290	7290
query15	230	168	168	168
query16	6612	461	456	456
query17	1397	580	590	580
query18	1608	310	328	310
query19	258	178	169	169
query20	118	109	110	109
query21	215	111	110	110
query22	4634	4364	4428	4364
query23	35157	34250	34190	34190
query24	9188	2726	2732	2726
query25	707	396	411	396
query26	1810	158	158	158
query27	2379	280	285	280
query28	6544	2474	2463	2463
query29	998	432	433	432
query30	331	166	167	166
query31	1025	782	815	782
query32	98	55	56	55
query33	670	291	298	291
query34	962	509	539	509
query35	956	753	759	753
query36	1113	951	949	949
query37	258	94	88	88
query38	4161	3934	3954	3934
query39	1504	1448	1422	1422
query40	278	102	100	100
query41	49	48	46	46
query42	128	98	102	98
query43	542	495	506	495
query44	1205	800	795	795
query45	202	167	167	167
query46	1155	717	705	705
query47	1977	1831	1857	1831
query48	412	327	333	327
query49	1254	449	424	424
query50	815	378	383	378
query51	7208	7013	7032	7013
query52	105	90	90	90
query53	256	180	178	178
query54	644	434	416	416
query55	78	77	77	77
query56	276	260	249	249
query57	1372	1159	1159	1159
query58	248	245	264	245
query59	3239	3200	3084	3084
query60	292	262	254	254
query61	104	103	107	103
query62	846	715	668	668
query63	217	185	181	181
query64	4440	645	605	605
query65	3379	3333	3250	3250
query66	1315	328	298	298
query67	16088	15534	15755	15534
query68	3562	597	560	560
query69	505	296	298	296
query70	1201	1121	1138	1121
query71	345	267	272	267
query72	6923	4005	4003	4003
query73	785	353	356	353
query74	9870	9030	9016	9016
query75	3455	2651	2693	2651
query76	2580	886	890	886
query77	448	297	301	297
query78	11056	9756	9782	9756
query79	1125	593	587	587
query80	719	450	448	448
query81	561	253	257	253
query82	625	139	134	134
query83	234	136	133	133
query84	255	71	67	67
query85	980	301	285	285
query86	346	307	293	293
query87	4484	4339	4319	4319
query88	3086	2208	2155	2155
query89	412	286	287	286
query90	1968	185	183	183
query91	142	99	101	99
query92	63	48	48	48
query93	1062	561	550	550
query94	804	287	285	285
query95	336	248	244	244
query96	614	273	282	273
query97	3309	3143	3127	3127
query98	223	203	193	193
query99	1513	1310	1298	1298
Total cold run time: 279333 ms
Total hot run time: 192758 ms

@doris-robot
Copy link

doris-robot commented Oct 18, 2024

ClickBench: Total hot run time: 32.47 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 10498e8d641e5e924121ba4c247b6c8adc92e60c, data reload: false

query1	0.04	0.04	0.04
query2	0.06	0.03	0.03
query3	0.23	0.06	0.06
query4	1.65	0.10	0.10
query5	0.52	0.51	0.49
query6	1.13	0.72	0.71
query7	0.03	0.02	0.02
query8	0.04	0.03	0.04
query9	0.56	0.50	0.53
query10	0.55	0.55	0.55
query11	0.14	0.10	0.10
query12	0.13	0.10	0.10
query13	0.61	0.59	0.60
query14	2.72	2.76	2.83
query15	0.88	0.81	0.84
query16	0.38	0.38	0.36
query17	1.05	1.00	1.02
query18	0.20	0.20	0.19
query19	1.91	1.75	1.91
query20	0.01	0.01	0.01
query21	15.36	0.61	0.62
query22	2.86	1.71	1.61
query23	16.96	0.93	0.84
query24	2.92	1.08	1.19
query25	0.11	0.09	0.24
query26	0.50	0.14	0.13
query27	0.05	0.05	0.05
query28	10.56	1.09	1.07
query29	12.58	3.21	3.21
query30	0.25	0.06	0.06
query31	2.89	0.38	0.37
query32	3.27	0.47	0.46
query33	3.01	3.00	3.05
query34	16.81	4.50	4.48
query35	4.54	4.44	4.52
query36	0.67	0.47	0.50
query37	0.08	0.06	0.06
query38	0.05	0.03	0.04
query39	0.03	0.03	0.02
query40	0.16	0.12	0.13
query41	0.09	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.66 s
Total hot run time: 32.47 s

liaoxin01
liaoxin01 approved these changes Oct 19, 2024
View reviewed changes
Copy link
Contributor

@liaoxin01 liaoxin01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yiguolei yiguolei merged commit ffbdfe2 into apache:master Oct 19, 2024
yiguolei added a commit to yiguolei/incubator-doris that referenced this pull request Oct 19, 2024
@yiguolei @Doris-Extras
[bugfix](handshake) brpc handshake should not use light pool (apache#…
120cc2c 
…42115)

The light pool may be full. Handshake is used to check the connection
state of brpc. Should not be interfered by the thread pool logic.

---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
yiguolei added a commit to yiguolei/incubator-doris that referenced this pull request Oct 19, 2024
@yiguolei @Doris-Extras
[bugfix](handshake) brpc handshake should not use light pool (apache#…
8119f8f 
…42115)

The light pool may be full. Handshake is used to check the connection
state of brpc. Should not be interfered by the thread pool logic.

---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
@yiguolei yiguolei mentioned this pull request Oct 19, 2024
Merged
yiguolei added a commit to yiguolei/incubator-doris that referenced this pull request Oct 19, 2024
@yiguolei @Doris-Extras
[bugfix](handshake) brpc handshake should not use light pool (apache#…
77bf470 
…42115)

## Proposed changes

The light pool may be full. Handshake is used to check the connection
state of brpc. Should not be interfered by the thread pool logic.

---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
@yiguolei yiguolei mentioned this pull request Oct 19, 2024
Merged
yiguolei added a commit that referenced this pull request Oct 19, 2024
@yiguolei @Doris-Extras
[bugfix](handshake) brpc handshake should not use light pool (#42115) (…
dc43864 
…#42127)

The light pool may be full. Handshake is used to check the connection
state of brpc. Should not be interfered by the thread pool logic.

---------
pick #42115

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->

Co-authored-by: yiguolei <yiguolei@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mrhhsg mrhhsg mrhhsg approved these changes

@liaoxin01 liaoxin01 liaoxin01 approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.7-merged dev/3.0.3-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@yiguolei @doris-robot @mrhhsg @liaoxin01 @Doris-Extras