Skip to content

[Fix]Fix publish may wait timeout because of dead BE #40763

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
yiguolei merged 1 commit into from
Sep 23, 2024
Merged

[Fix]Fix publish may wait timeout because of dead BE #40763

yiguolei merged 1 commit into from
Sep 23, 2024

Conversation

@wangbo
Copy link
Contributor

@wangbo wangbo commented Sep 12, 2024 •
edited
Loading

Proposed changes

AckResponseHandler should only accept alive BE, otherwise pubslih may wait timeout if dead Be exists.

@doris-robot
Copy link

doris-robot commented Sep 12, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@wangbo
Copy link
Contributor Author

wangbo commented Sep 12, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Sep 12, 2024

TPC-H: Total hot run time: 43097 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6cd99ec36f844a510f4b71c052c6b31cf06c5fe8, data reload: false

------ Round 1 ----------------------------------
q1	18701	7465	7252	7252
q2	2245	191	202	191
q3	11383	1375	1372	1372
q4	10256	991	1068	991
q5	8092	3244	3183	3183
q6	240	155	154	154
q7	1043	645	620	620
q8	9448	2009	2013	2009
q9	6748	6298	6295	6295
q10	7032	2505	2505	2505
q11	437	250	255	250
q12	409	230	228	228
q13	17761	3011	3013	3011
q14	291	256	247	247
q15	578	520	516	516
q16	501	425	418	418
q17	976	943	948	943
q18	7281	6899	6861	6861
q19	1383	1236	1227	1227
q20	609	347	331	331
q21	3859	3515	3524	3515
q22	1083	978	1016	978
Total cold run time: 110356 ms
Total hot run time: 43097 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7136	7139	7178	7139
q2	338	238	228	228
q3	2881	2907	2899	2899
q4	1980	1970	1965	1965
q5	5468	5422	5403	5403
q6	231	147	144	144
q7	2040	1716	1703	1703
q8	3225	3296	3268	3268
q9	8407	8392	8336	8336
q10	3416	3457	3428	3428
q11	596	483	467	467
q12	794	585	590	585
q13	5587	3079	3020	3020
q14	319	275	272	272
q15	575	529	525	525
q16	497	448	442	442
q17	1733	1735	1701	1701
q18	8062	7577	7768	7577
q19	1719	1685	1681	1681
q20	2033	1803	1820	1803
q21	5648	5414	5468	5414
q22	1115	1034	999	999
Total cold run time: 63800 ms
Total hot run time: 58999 ms

@doris-robot
Copy link

doris-robot commented Sep 12, 2024

TPC-DS: Total hot run time: 195726 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6cd99ec36f844a510f4b71c052c6b31cf06c5fe8, data reload: false

query1	930	386	391	386
query2	6492	1787	1836	1787
query3	6664	214	220	214
query4	25919	24138	24127	24127
query5	5387	547	535	535
query6	269	176	161	161
query7	4582	305	315	305
query8	274	219	219	219
query9	8496	2633	2618	2618
query10	508	291	282	282
query11	16227	15712	15715	15712
query12	161	106	105	105
query13	1695	394	386	386
query14	11013	6613	7139	6613
query15	217	170	177	170
query16	7573	468	459	459
query17	1546	601	555	555
query18	1962	290	282	282
query19	190	151	144	144
query20	118	112	111	111
query21	205	106	102	102
query22	4629	4335	4127	4127
query23	34734	33939	33661	33661
query24	9665	3113	3133	3113
query25	647	402	381	381
query26	1151	157	159	157
query27	2695	280	294	280
query28	6843	2100	2095	2095
query29	928	414	412	412
query30	295	164	150	150
query31	984	767	796	767
query32	112	58	57	57
query33	727	294	303	294
query34	912	489	492	489
query35	881	748	745	745
query36	1052	901	885	885
query37	165	79	81	79
query38	4135	3858	3883	3858
query39	1496	1410	1409	1409
query40	207	116	118	116
query41	50	48	48	48
query42	122	102	97	97
query43	492	457	460	457
query44	1265	813	790	790
query45	199	180	173	173
query46	1117	811	807	807
query47	1873	1768	1789	1768
query48	368	296	306	296
query49	1105	467	449	449
query50	936	446	444	444
query51	7041	7060	7050	7050
query52	103	88	88	88
query53	257	185	190	185
query54	798	466	478	466
query55	79	79	82	79
query56	301	277	258	258
query57	1203	1090	1084	1084
query58	254	257	234	234
query59	2843	2870	2595	2595
query60	301	272	282	272
query61	105	105	106	105
query62	911	661	668	661
query63	226	187	190	187
query64	4617	682	688	682
query65	3244	3204	3169	3169
query66	1413	300	306	300
query67	15989	15659	15578	15578
query68	3151	783	592	592
query69	414	272	278	272
query70	1161	1081	1065	1065
query71	333	283	286	283
query72	5985	3991	4046	3991
query73	757	336	337	336
query74	9206	9261	9166	9166
query75	3365	2728	2691	2691
query76	1900	1311	1292	1292
query77	439	321	321	321
query78	9747	9430	9936	9430
query79	914	885	875	875
query80	883	845	844	844
query81	454	268	283	268
query82	267	267	265	265
query83	201	191	205	191
query84	238	114	113	113
query85	782	539	430	430
query86	319	309	299	299
query87	4338	4405	4316	4316
query88	4218	4163	4150	4150
query89	380	369	371	369
query90	1548	328	320	320
query91	129	129	126	126
query92	76	76	75	75
query93	1047	1069	1058	1058
query94	722	404	369	369
query95	499	425	425	425
query96	479	479	478	478
query97	3123	3110	3123	3110
query98	241	241	233	233
query99	1560	1316	1303	1303
Total cold run time: 278774 ms
Total hot run time: 195726 ms

@doris-robot
Copy link

doris-robot commented Sep 12, 2024

ClickBench: Total hot run time: 31.04 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6cd99ec36f844a510f4b71c052c6b31cf06c5fe8, data reload: false

query1	0.04	0.05	0.04
query2	0.07	0.04	0.04
query3	0.23	0.05	0.05
query4	1.70	0.07	0.06
query5	0.50	0.50	0.55
query6	1.12	0.72	0.73
query7	0.02	0.01	0.02
query8	0.05	0.05	0.05
query9	0.57	0.53	0.53
query10	0.59	0.57	0.56
query11	0.16	0.12	0.12
query12	0.16	0.12	0.12
query13	0.63	0.61	0.64
query14	1.48	1.48	1.49
query15	0.89	0.87	0.88
query16	0.36	0.35	0.37
query17	1.01	1.05	1.03
query18	0.22	0.21	0.21
query19	1.96	1.90	1.85
query20	0.02	0.01	0.01
query21	15.40	0.66	0.66
query22	4.07	7.58	1.17
query23	17.94	1.34	1.31
query24	2.08	0.24	0.23
query25	0.18	0.08	0.08
query26	0.28	0.18	0.17
query27	0.09	0.08	0.07
query28	13.18	1.13	1.09
query29	12.56	3.34	3.35
query30	0.24	0.06	0.05
query31	2.87	0.42	0.42
query32	3.24	0.49	0.49
query33	3.03	3.02	3.09
query34	15.50	4.35	4.33
query35	4.35	4.33	4.35
query36	0.68	0.47	0.50
query37	0.19	0.17	0.16
query38	0.15	0.14	0.14
query39	0.05	0.04	0.04
query40	0.17	0.14	0.13
query41	0.10	0.05	0.05
query42	0.06	0.05	0.04
query43	0.05	0.04	0.04
Total cold run time: 108.24 s
Total hot run time: 31.04 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 23, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Sep 23, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 23, 2024

PR approved by anyone and no changes requested.

xinyiZzz
xinyiZzz approved these changes Sep 23, 2024
View reviewed changes
Copy link
Contributor

@xinyiZzz xinyiZzz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yiguolei yiguolei merged commit f57dc92 into apache:master Sep 23, 2024
dataroaring pushed a commit that referenced this pull request Oct 9, 2024
@wangbo @dataroaring
[Fix]Fix publish may wait timeout because of dead BE (#40763)
e184f04 
## Proposed changes
```AckResponseHandler``` should only accept alive BE, otherwise pubslih may wait timeout if dead Be exists.
wangbo added a commit to wangbo/incubator-doris that referenced this pull request Nov 25, 2024
@wangbo
[Fix]Fix publish may wait timeout because of dead BE (apache#40763)
52ff294 
```AckResponseHandler``` should only accept alive BE, otherwise pubslih may wait timeout if dead Be exists.
@wangbo wangbo mentioned this pull request Nov 25, 2024
Merged
16 tasks
wangbo added a commit that referenced this pull request Nov 25, 2024
@wangbo
[branch-2.1] pick some pr from master (#44556)
5eb433e 
pick #40468 #40535  #40763 #40855
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yiguolei yiguolei yiguolei approved these changes

@xinyiZzz xinyiZzz xinyiZzz approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.x dev/2.1.8-merged dev/3.0.3-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@wangbo @doris-robot @yiguolei @xinyiZzz @dataroaring