Skip to content

[fix](fe) fix start fe failed when recover fe in different host #35612

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
924060929 merged 1 commit into from
May 30, 2024
Merged

[fix](fe) fix start fe failed when recover fe in different host #35612

924060929 merged 1 commit into from
May 30, 2024

Conversation

@924060929
Copy link
Contributor

@924060929 924060929 commented May 29, 2024 •
edited
Loading

Proposed changes

An exception introduced by #35203 cause the frontend start failed

In this cases:

  1. all frontends are down and recover frontend in different host by the original metadata
  2. or the only frontend's ip changed, and start with -r option to recover from original metadata

@doris-robot
Copy link

doris-robot commented May 29, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@924060929
Copy link
Contributor Author

924060929 commented May 29, 2024

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented May 29, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 29, 2024
@github-actions
Copy link
Contributor

github-actions bot commented May 29, 2024

PR approved by anyone and no changes requested.

@924060929 924060929 changed the title [fix](fe) fix start fe failed when start fe in different host [fix](fe) fix start fe failed when recover fe in different host May 29, 2024
@doris-robot
Copy link

doris-robot commented May 29, 2024

TPC-H: Total hot run time: 41675 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bbd29a67a157cfe83e2cf22f63bbd3b2a8b6172c, data reload: false

------ Round 1 ----------------------------------
q1	17611	4345	4248	4248
q2	2019	201	192	192
q3	10469	1188	1277	1188
q4	10207	847	873	847
q5	7476	2765	2754	2754
q6	229	129	137	129
q7	976	624	605	605
q8	9220	2174	2144	2144
q9	9398	6752	6755	6752
q10	9186	3808	3894	3808
q11	458	248	240	240
q12	434	226	224	224
q13	17282	3176	3246	3176
q14	276	226	239	226
q15	513	474	475	474
q16	476	385	374	374
q17	996	752	783	752
q18	8468	7930	7965	7930
q19	6615	1622	1639	1622
q20	652	316	314	314
q21	5167	3350	4067	3350
q22	387	326	337	326
Total cold run time: 118515 ms
Total hot run time: 41675 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4555	4439	4416	4416
q2	371	267	257	257
q3	3180	2940	2965	2940
q4	1960	1609	1661	1609
q5	5444	5527	5526	5526
q6	220	124	129	124
q7	2225	1790	1820	1790
q8	3234	3430	3433	3430
q9	8692	8758	8736	8736
q10	4028	3721	3818	3721
q11	599	500	490	490
q12	847	628	620	620
q13	17097	3165	3152	3152
q14	309	272	270	270
q15	534	483	476	476
q16	506	452	429	429
q17	1822	1502	1499	1499
q18	7793	7662	7443	7443
q19	1715	1582	1595	1582
q20	1988	1785	1776	1776
q21	9919	4850	4599	4599
q22	594	542	572	542
Total cold run time: 77632 ms
Total hot run time: 55427 ms

@doris-robot
Copy link

doris-robot commented May 29, 2024

TPC-DS: Total hot run time: 169356 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bbd29a67a157cfe83e2cf22f63bbd3b2a8b6172c, data reload: false

query1	924	383	382	382
query2	6452	2475	2290	2290
query3	6645	212	219	212
query4	19743	17600	17277	17277
query5	4172	426	421	421
query6	250	159	153	153
query7	4588	300	283	283
query8	322	297	275	275
query9	8448	2402	2372	2372
query10	455	286	260	260
query11	10568	10250	10049	10049
query12	142	101	91	91
query13	1658	358	360	358
query14	10353	7047	7029	7029
query15	227	196	189	189
query16	7746	260	257	257
query17	1634	523	499	499
query18	1905	263	269	263
query19	198	154	159	154
query20	92	86	88	86
query21	203	132	127	127
query22	4136	3905	3856	3856
query23	33457	33145	33001	33001
query24	8663	2798	2850	2798
query25	559	348	361	348
query26	701	155	154	154
query27	2229	325	321	321
query28	5739	2076	2074	2074
query29	882	612	594	594
query30	239	154	160	154
query31	1006	742	752	742
query32	86	52	55	52
query33	634	270	263	263
query34	866	484	466	466
query35	711	596	606	596
query36	1076	942	910	910
query37	105	72	70	70
query38	2908	2766	2769	2766
query39	838	787	784	784
query40	203	132	128	128
query41	57	53	53	53
query42	135	97	94	94
query43	578	567	546	546
query44	1087	740	741	740
query45	190	171	169	169
query46	1084	734	718	718
query47	1869	1755	1767	1755
query48	380	287	291	287
query49	844	377	384	377
query50	776	387	382	382
query51	6940	6635	6813	6635
query52	100	90	85	85
query53	352	296	299	296
query54	758	432	421	421
query55	76	71	72	71
query56	254	240	240	240
query57	1102	1033	1021	1021
query58	232	217	242	217
query59	3537	3415	3169	3169
query60	280	272	261	261
query61	87	84	84	84
query62	593	444	448	444
query63	316	291	285	285
query64	8499	2316	1727	1727
query65	3156	3094	3112	3094
query66	793	330	333	330
query67	15379	15050	15011	15011
query68	4514	531	519	519
query69	446	271	265	265
query70	1191	1128	1148	1128
query71	365	269	265	265
query72	7728	2708	2530	2530
query73	714	321	319	319
query74	6036	5594	5594	5594
query75	3359	2624	2680	2624
query76	2331	995	1010	995
query77	379	271	268	268
query78	10307	9730	9691	9691
query79	1440	519	519	519
query80	1051	447	433	433
query81	527	225	227	225
query82	943	91	93	91
query83	239	173	168	168
query84	241	88	87	87
query85	1109	254	254	254
query86	383	308	306	306
query87	3370	3075	3127	3075
query88	3064	2347	2352	2347
query89	495	403	382	382
query90	1951	196	191	191
query91	133	108	110	108
query92	65	53	49	49
query93	1039	510	502	502
query94	1309	189	182	182
query95	393	306	304	304
query96	576	265	268	265
query97	3200	3018	3021	3018
query98	246	215	207	207
query99	1188	829	868	829
Total cold run time: 261549 ms
Total hot run time: 169356 ms

@doris-robot
Copy link

doris-robot commented May 29, 2024

ClickBench: Total hot run time: 30.26 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bbd29a67a157cfe83e2cf22f63bbd3b2a8b6172c, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.03	0.04
query3	0.23	0.05	0.05
query4	1.67	0.10	0.08
query5	0.51	0.48	0.52
query6	1.12	0.71	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.53	0.49	0.50
query10	0.54	0.55	0.55
query11	0.15	0.11	0.11
query12	0.15	0.12	0.11
query13	0.59	0.58	0.59
query14	0.80	0.79	0.78
query15	0.82	0.81	0.80
query16	0.37	0.36	0.37
query17	0.95	0.94	0.95
query18	0.20	0.26	0.24
query19	1.81	1.73	1.65
query20	0.01	0.02	0.01
query21	15.75	0.66	0.66
query22	4.75	7.33	1.73
query23	18.21	1.38	1.24
query24	1.53	0.29	0.22
query25	0.14	0.08	0.08
query26	0.26	0.17	0.17
query27	0.08	0.07	0.08
query28	13.50	1.02	0.99
query29	13.25	3.34	3.30
query30	0.23	0.06	0.05
query31	2.87	0.40	0.37
query32	3.27	0.46	0.47
query33	2.88	2.92	2.89
query34	17.19	4.48	4.41
query35	4.47	4.52	4.52
query36	0.70	0.50	0.47
query37	0.18	0.14	0.15
query38	0.17	0.14	0.14
query39	0.04	0.04	0.03
query40	0.16	0.14	0.14
query41	0.08	0.04	0.04
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 110.45 s
Total hot run time: 30.26 s

@924060929 924060929 merged commit bd82026 into apache:master May 30, 2024
@924060929 924060929 deleted the fix_recover_fe branch May 30, 2024 03:36
dataroaring pushed a commit that referenced this pull request May 31, 2024
@924060929 @dataroaring
[fix](fe) fix start fe failed when recover fe in different host (#35612)
1db35a5 
An exception introduced by #35203 cause the frontend start failed

In this cases:
1. all frontends are down and recover frontend in different host by the original metadata
2. or the only frontend's ip changed, and start with `-r` option to recover from original metadata
@morrySnow morrySnow added not-merge/2.0 do not merge into 2.0 branch not-merge/2.1 and removed dev/2.0.x dev/2.1.x labels Jun 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morrySnow morrySnow morrySnow approved these changes

+1 more reviewer

@swjtu-zhanglei swjtu-zhanglei swjtu-zhanglei approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.0-merged not-merge/2.0 do not merge into 2.0 branch not-merge/2.1 reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@924060929 @doris-robot @swjtu-zhanglei @morrySnow @dataroaring