Skip to content

branch-3.1: [bugfix](k8s) using stdout to output #54129 #54172

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morrySnow merged 1 commit into from
Aug 1, 2025
Merged

branch-3.1: [bugfix](k8s) using stdout to output #54129 #54172

morrySnow merged 1 commit into from
Aug 1, 2025

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Aug 1, 2025

Cherry-picked from #54129

@yiguolei
[bugfix](k8s) using stdout to output (#54129)
81bce50 
### What problem does this PR solve?

K8S using stdout to capture BE INFO logs. But when BE crash be will try
to output crash stack to be.out file. In the past, it could not output
to be.out file. In this feature, I format the behaviors of be log:
1. -- daemon, output log to be.info file and output crash stack to
be.out file.
2. --console,
2.1 enable_file_logger = true, output all logs to console (stdout) and
output all logs to be.info and crash stack to be.out
2.2 enable_file_logger = false, output all logs to console(stdout) and
crash stack to be.out

we will always output crash stack to logs/be.out file.
@github-actions github-actions bot requested a review from morrySnow as a code owner August 1, 2025 01:09
@Thearas
Copy link
Contributor

Thearas commented Aug 1, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring closed this Aug 1, 2025
@dataroaring dataroaring reopened this Aug 1, 2025
@Thearas
Copy link
Contributor

Thearas commented Aug 1, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Aug 1, 2025

TPC-H: Total hot run time: 32068 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 81bce50f4b8a2df97340dcbcffe9f15eb51c11c7, data reload: false

------ Round 1 ----------------------------------
q1	17580	5421	5383	5383
q2	2048	282	162	162
q3	10594	1188	718	718
q4	10258	859	437	437
q5	8432	2291	2074	2074
q6	179	164	133	133
q7	886	736	605	605
q8	9323	1395	1150	1150
q9	5213	4901	4936	4901
q10	6786	2249	1789	1789
q11	478	274	250	250
q12	326	351	212	212
q13	17795	3580	3025	3025
q14	223	225	202	202
q15	531	478	466	466
q16	416	422	370	370
q17	585	837	360	360
q18	6853	6339	6226	6226
q19	1472	946	533	533
q20	316	350	203	203
q21	2688	2086	1894	1894
q22	1040	1019	975	975
Total cold run time: 104022 ms
Total hot run time: 32068 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5571	5505	5463	5463
q2	235	333	230	230
q3	2200	2614	2336	2336
q4	1373	1799	1354	1354
q5	4359	4791	4848	4791
q6	165	158	130	130
q7	1999	1929	1787	1787
q8	2590	2801	2661	2661
q9	7251	7238	7283	7238
q10	3068	3336	2866	2866
q11	579	533	498	498
q12	686	739	579	579
q13	3381	3884	3246	3246
q14	275	304	284	284
q15	518	477	466	466
q16	418	491	414	414
q17	1203	1754	1230	1230
q18	7547	7335	7243	7243
q19	820	1134	1070	1070
q20	1992	2006	1920	1920
q21	5247	4952	4596	4596
q22	1125	1089	1050	1050
Total cold run time: 52602 ms
Total hot run time: 51452 ms

@doris-robot
Copy link

doris-robot commented Aug 1, 2025

BE UT Coverage Report

Increment line coverage 0.00% (0/35) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 45.40% (12632/27821)
Line Coverage 36.25% (112603/310597)
Region Coverage 35.29% (58185/164872)
Branch Coverage 32.49% (31642/97400)

@doris-robot
Copy link

doris-robot commented Aug 1, 2025

TPC-DS: Total hot run time: 197422 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 81bce50f4b8a2df97340dcbcffe9f15eb51c11c7, data reload: false

query1	1307	953	897	897
query2	6241	1949	1959	1949
query3	10956	4383	4540	4383
query4	32811	23764	23356	23356
query5	3931	587	468	468
query6	269	183	185	183
query7	3986	494	332	332
query8	320	275	237	237
query9	9588	2599	2600	2599
query10	467	337	257	257
query11	18319	15430	15300	15300
query12	158	104	104	104
query13	1568	530	424	424
query14	9787	7573	7177	7177
query15	224	212	195	195
query16	7933	649	512	512
query17	1582	753	613	613
query18	2124	416	322	322
query19	211	197	188	188
query20	130	116	122	116
query21	205	128	112	112
query22	4535	4489	4376	4376
query23	34681	34140	34231	34140
query24	7256	2670	2733	2670
query25	496	473	410	410
query26	1170	276	169	169
query27	2091	475	364	364
query28	5234	2200	2212	2200
query29	716	581	450	450
query30	235	185	167	167
query31	991	919	838	838
query32	82	60	55	55
query33	484	358	312	312
query34	772	854	538	538
query35	781	812	720	720
query36	1009	1036	948	948
query37	105	92	69	69
query38	4096	4071	4018	4018
query39	1521	1468	1486	1468
query40	211	118	104	104
query41	49	48	46	46
query42	120	104	101	101
query43	524	520	491	491
query44	1340	834	837	834
query45	185	178	166	166
query46	871	1047	681	681
query47	1936	1942	1917	1917
query48	414	411	342	342
query49	761	484	411	411
query50	677	702	436	436
query51	7423	7336	7239	7239
query52	102	106	91	91
query53	228	253	189	189
query54	548	562	464	464
query55	84	81	80	80
query56	286	272	253	253
query57	1300	1256	1218	1218
query58	236	220	211	211
query59	3045	3271	3116	3116
query60	298	273	267	267
query61	112	122	110	110
query62	789	789	697	697
query63	228	194	198	194
query64	4326	980	671	671
query65	3333	3303	3347	3303
query66	1029	412	300	300
query67	16320	15817	15498	15498
query68	3205	868	568	568
query69	482	299	259	259
query70	1196	1097	1142	1097
query71	380	302	259	259
query72	5696	3808	3952	3808
query73	624	750	356	356
query74	10091	9174	9134	9134
query75	3170	3191	2660	2660
query76	3036	1176	784	784
query77	498	357	293	293
query78	10263	10341	9669	9669
query79	4115	855	598	598
query80	1759	513	435	435
query81	552	267	217	217
query82	1136	117	88	88
query83	252	159	141	141
query84	241	103	81	81
query85	803	363	290	290
query86	470	309	285	285
query87	4303	4336	4275	4275
query88	5154	2390	2395	2390
query89	414	333	294	294
query90	1754	192	192	192
query91	142	148	108	108
query92	62	56	52	52
query93	3002	870	546	546
query94	754	406	306	306
query95	340	268	270	268
query96	490	607	282	282
query97	3208	3257	3137	3137
query98	228	212	204	204
query99	1321	1410	1280	1280
Total cold run time: 294288 ms
Total hot run time: 197422 ms

@doris-robot
Copy link

doris-robot commented Aug 1, 2025

ClickBench: Total hot run time: 28.78 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 81bce50f4b8a2df97340dcbcffe9f15eb51c11c7, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.04	0.03
query3	0.24	0.06	0.06
query4	1.63	0.11	0.10
query5	0.53	0.51	0.50
query6	1.13	0.73	0.73
query7	0.02	0.02	0.02
query8	0.05	0.04	0.03
query9	0.58	0.50	0.50
query10	0.54	0.55	0.54
query11	0.14	0.10	0.11
query12	0.13	0.11	0.10
query13	0.62	0.60	0.60
query14	0.79	0.80	0.79
query15	0.84	0.83	0.82
query16	0.38	0.36	0.39
query17	1.07	1.01	1.09
query18	0.23	0.23	0.22
query19	1.98	1.78	1.82
query20	0.02	0.01	0.02
query21	15.38	0.92	0.56
query22	0.73	1.03	0.61
query23	14.92	1.44	0.55
query24	3.42	0.98	2.11
query25	0.17	0.09	0.19
query26	0.27	0.15	0.13
query27	0.05	0.06	0.06
query28	14.18	0.92	0.43
query29	12.62	3.91	3.22
query30	0.25	0.09	0.06
query31	2.82	0.60	0.40
query32	3.22	0.53	0.48
query33	2.97	2.98	3.04
query34	16.66	5.12	4.59
query35	4.59	4.58	4.53
query36	0.66	0.49	0.47
query37	0.09	0.06	0.06
query38	0.04	0.04	0.03
query39	0.04	0.02	0.02
query40	0.16	0.14	0.13
query41	0.08	0.03	0.03
query42	0.03	0.03	0.02
query43	0.04	0.03	0.02
Total cold run time: 104.4 s
Total hot run time: 28.78 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 1, 2025

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/35) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 76.32% (20979/27488)
Line Coverage 69.69% (216322/310389)
Region Coverage 67.71% (129476/191211)
Branch Coverage 61.29% (67373/109924)

@morrySnow morrySnow merged commit ff8868f into branch-3.1 Aug 1, 2025
21 of 23 checks passed
@github-actions github-actions bot deleted the auto-pick-54129-branch-3.1 branch August 1, 2025 03:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@Thearas @doris-robot @hello-stephen @morrySnow @dataroaring @yiguolei