Skip to content

[improve](job) Adding FailureReason after manually changing the status of a streaming job #57551

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
JNSimba merged 2 commits into from
Nov 3, 2025
Merged

[improve](job) Adding FailureReason after manually changing the status of a streaming job #57551

JNSimba merged 2 commits into from
Nov 3, 2025

Conversation

@JNSimba
Copy link
Member

@JNSimba JNSimba commented Oct 31, 2025

What problem does this PR solve?

Adding FailureReason after manually changing the status of a streaming job

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 31, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@JNSimba
Copy link
Member Author

JNSimba commented Oct 31, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Oct 31, 2025

TPC-DS: Total hot run time: 190622 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fd82747f287f882b2d17f3fab0f4353a68e4ac0f, data reload: false

query1	1043	413	425	413
query2	6563	1796	1829	1796
query3	6768	235	229	229
query4	26621	24388	23550	23550
query5	5892	673	485	485
query6	406	250	234	234
query7	4654	533	321	321
query8	346	276	259	259
query9	8746	2799	2798	2798
query10	580	371	320	320
query11	15516	15173	14961	14961
query12	202	125	123	123
query13	1710	630	467	467
query14	12160	9599	9447	9447
query15	250	195	181	181
query16	7820	681	557	557
query17	1703	871	747	747
query18	2243	481	366	366
query19	323	246	176	176
query20	135	142	127	127
query21	218	154	117	117
query22	4680	4819	4308	4308
query23	34806	34012	33407	33407
query24	8407	2478	2510	2478
query25	590	526	484	484
query26	1324	325	171	171
query27	3562	532	370	370
query28	4452	2252	2230	2230
query29	917	659	506	506
query30	303	221	220	220
query31	968	852	831	831
query32	89	74	76	74
query33	582	376	325	325
query34	813	897	538	538
query35	841	861	808	808
query36	977	1089	918	918
query37	122	105	87	87
query38	3649	3708	3638	3638
query39	1510	1466	1406	1406
query40	221	125	116	116
query41	60	57	56	56
query42	117	105	108	105
query43	479	510	470	470
query44	1226	747	720	720
query45	184	182	186	182
query46	880	989	637	637
query47	1723	1866	1673	1673
query48	399	412	308	308
query49	787	495	408	408
query50	643	694	424	424
query51	3911	4037	3869	3869
query52	113	106	97	97
query53	224	263	199	199
query54	596	585	515	515
query55	84	89	93	89
query56	341	330	330	330
query57	1183	1176	1092	1092
query58	286	274	271	271
query59	2544	2640	2532	2532
query60	345	344	326	326
query61	157	155	157	155
query62	793	720	670	670
query63	227	187	184	184
query64	4440	1161	837	837
query65	4011	3924	3932	3924
query66	1024	429	342	342
query67	15415	15196	15213	15196
query68	8237	877	599	599
query69	496	327	292	292
query70	1336	1313	1172	1172
query71	471	340	327	327
query72	5964	4877	4993	4877
query73	647	616	358	358
query74	8926	9175	8956	8956
query75	3312	3355	2798	2798
query76	3124	1152	724	724
query77	522	397	330	330
query78	9554	9691	8976	8976
query79	2289	829	597	597
query80	615	561	471	471
query81	513	262	235	235
query82	483	156	134	134
query83	261	265	245	245
query84	259	107	93	93
query85	891	510	423	423
query86	334	319	287	287
query87	3690	3724	3658	3658
query88	3843	2217	2189	2189
query89	390	321	296	296
query90	2191	213	212	212
query91	162	161	139	139
query92	80	64	61	61
query93	2021	958	628	628
query94	752	435	331	331
query95	381	316	310	310
query96	485	565	278	278
query97	2969	2977	2891	2891
query98	258	215	211	211
query99	1364	1446	1324	1324
Total cold run time: 281724 ms
Total hot run time: 190622 ms

@doris-robot
Copy link

doris-robot commented Oct 31, 2025

ClickBench: Total hot run time: 28.06 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fd82747f287f882b2d17f3fab0f4353a68e4ac0f, data reload: false

query1	0.05	0.05	0.04
query2	0.10	0.05	0.05
query3	0.26	0.08	0.09
query4	1.60	0.11	0.12
query5	0.27	0.26	0.25
query6	1.15	0.66	0.63
query7	0.03	0.02	0.03
query8	0.06	0.04	0.04
query9	0.60	0.53	0.52
query10	0.57	0.58	0.57
query11	0.17	0.12	0.11
query12	0.15	0.13	0.12
query13	0.61	0.60	0.60
query14	1.02	1.01	1.01
query15	0.85	0.84	0.84
query16	0.40	0.38	0.40
query17	0.98	1.06	1.06
query18	0.22	0.23	0.20
query19	1.95	1.87	1.81
query20	0.02	0.01	0.02
query21	15.42	0.20	0.13
query22	4.98	0.06	0.04
query23	15.63	0.26	0.10
query24	2.88	1.22	0.79
query25	0.08	0.06	0.07
query26	0.14	0.13	0.13
query27	0.07	0.06	0.05
query28	4.74	1.13	0.92
query29	12.58	3.98	3.29
query30	0.27	0.14	0.12
query31	2.81	0.58	0.37
query32	3.23	0.54	0.47
query33	3.07	3.20	3.16
query34	16.01	5.24	4.57
query35	4.63	4.60	4.57
query36	0.69	0.51	0.49
query37	0.10	0.07	0.07
query38	0.07	0.04	0.04
query39	0.03	0.03	0.03
query40	0.18	0.14	0.13
query41	0.08	0.04	0.03
query42	0.04	0.02	0.02
query43	0.04	0.03	0.04
Total cold run time: 98.83 s
Total hot run time: 28.06 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 31, 2025

FE Regression Coverage Report

Increment line coverage 100.00% (9/9) 🎉
Increment coverage report
Complete coverage report

sollhui
sollhui approved these changes Oct 31, 2025
View reviewed changes
Copy link
Contributor

@sollhui sollhui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

github-actions bot commented Oct 31, 2025

PR approved by anyone and no changes requested.

sollhui
sollhui approved these changes Oct 31, 2025
View reviewed changes
Copy link
Contributor

@sollhui sollhui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@JNSimba
Copy link
Member Author

JNSimba commented Oct 31, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Oct 31, 2025

TPC-DS: Total hot run time: 190443 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 912becf5c0a7b8d1c4fd4513011a7c23c299eefe, data reload: false

query1	1042	429	390	390
query2	6550	1716	1687	1687
query3	6752	222	218	218
query4	26592	23590	23329	23329
query5	4975	650	486	486
query6	351	247	231	231
query7	4652	510	314	314
query8	307	266	272	266
query9	8739	2589	2619	2589
query10	535	346	304	304
query11	15964	15230	14890	14890
query12	204	125	117	117
query13	1703	583	442	442
query14	12739	9498	9428	9428
query15	268	191	182	182
query16	7829	707	539	539
query17	1641	859	715	715
query18	2258	476	374	374
query19	324	254	223	223
query20	140	138	133	133
query21	224	144	124	124
query22	4839	4791	4672	4672
query23	34580	33695	33928	33695
query24	8218	2514	2582	2514
query25	609	520	468	468
query26	1943	298	169	169
query27	2888	528	375	375
query28	4482	2238	2230	2230
query29	821	685	504	504
query30	299	239	201	201
query31	1000	868	783	783
query32	84	75	75	75
query33	613	385	339	339
query34	846	915	557	557
query35	873	864	803	803
query36	960	1040	936	936
query37	129	121	93	93
query38	3750	3586	3553	3553
query39	1459	1397	1395	1395
query40	215	124	115	115
query41	58	55	56	55
query42	125	113	110	110
query43	489	486	482	482
query44	1268	728	735	728
query45	184	178	167	167
query46	894	994	646	646
query47	1795	1804	1734	1734
query48	406	450	323	323
query49	774	520	410	410
query50	683	693	405	405
query51	3913	3991	3879	3879
query52	105	114	103	103
query53	243	272	196	196
query54	303	288	278	278
query55	85	88	85	85
query56	332	306	290	290
query57	1166	1196	1128	1128
query58	288	280	275	275
query59	2608	2618	2526	2526
query60	347	339	327	327
query61	160	154	160	154
query62	796	707	672	672
query63	233	198	196	196
query64	4415	1187	897	897
query65	4056	3934	3948	3934
query66	1082	418	332	332
query67	15514	15267	15061	15061
query68	7222	874	594	594
query69	458	329	295	295
query70	1299	1181	1271	1181
query71	463	336	314	314
query72	5821	5043	4964	4964
query73	538	601	355	355
query74	9205	9068	8945	8945
query75	3773	3357	2857	2857
query76	3104	1166	743	743
query77	781	407	320	320
query78	9580	9486	8896	8896
query79	2686	850	608	608
query80	696	571	508	508
query81	501	262	227	227
query82	437	161	139	139
query83	273	263	251	251
query84	259	110	97	97
query85	912	482	436	436
query86	389	319	287	287
query87	3823	3700	3667	3667
query88	4167	2256	2252	2252
query89	388	321	290	290
query90	1950	217	224	217
query91	168	166	135	135
query92	81	69	61	61
query93	2571	950	639	639
query94	656	464	351	351
query95	408	329	302	302
query96	496	579	290	290
query97	2928	2980	2873	2873
query98	237	218	214	214
query99	1326	1424	1284	1284
Total cold run time: 282315 ms
Total hot run time: 190443 ms

@doris-robot
Copy link

doris-robot commented Oct 31, 2025

ClickBench: Total hot run time: 27.53 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 912becf5c0a7b8d1c4fd4513011a7c23c299eefe, data reload: false

query1	0.06	0.05	0.05
query2	0.09	0.05	0.05
query3	0.25	0.08	0.08
query4	1.60	0.11	0.11
query5	0.27	0.26	0.26
query6	1.18	0.66	0.65
query7	0.03	0.04	0.02
query8	0.06	0.04	0.04
query9	0.60	0.52	0.51
query10	0.58	0.57	0.57
query11	0.17	0.11	0.11
query12	0.16	0.12	0.13
query13	0.62	0.60	0.61
query14	1.01	1.03	1.00
query15	0.85	0.84	0.84
query16	0.39	0.39	0.40
query17	1.03	0.99	1.04
query18	0.25	0.20	0.20
query19	1.89	1.79	1.82
query20	0.02	0.01	0.02
query21	15.46	0.21	0.12
query22	5.03	0.06	0.05
query23	15.70	0.25	0.10
query24	2.87	0.52	0.44
query25	0.07	0.06	0.05
query26	0.14	0.14	0.14
query27	0.06	0.05	0.06
query28	4.01	1.15	0.93
query29	12.56	3.94	3.25
query30	0.28	0.14	0.11
query31	2.82	0.59	0.39
query32	3.24	0.55	0.47
query33	3.10	3.02	3.03
query34	15.97	5.12	4.58
query35	4.62	4.63	4.61
query36	0.66	0.51	0.49
query37	0.10	0.07	0.06
query38	0.07	0.04	0.04
query39	0.04	0.03	0.03
query40	0.18	0.13	0.14
query41	0.09	0.03	0.02
query42	0.04	0.04	0.03
query43	0.04	0.04	0.03
Total cold run time: 98.26 s
Total hot run time: 27.53 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 31, 2025

FE UT Coverage Report

Increment line coverage 0.00% (0/13) 🎉
Increment coverage report
Complete coverage report

@JNSimba
Copy link
Member Author

JNSimba commented Oct 31, 2025

run p0

@JNSimba
Copy link
Member Author

JNSimba commented Oct 31, 2025

run cloud_p0

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 31, 2025

FE Regression Coverage Report

Increment line coverage 84.62% (11/13) 🎉
Increment coverage report
Complete coverage report

@JNSimba
Copy link
Member Author

JNSimba commented Oct 31, 2025

run p0

liaoxin01
liaoxin01 approved these changes Nov 3, 2025
View reviewed changes
Copy link
Contributor

@liaoxin01 liaoxin01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 3, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Nov 3, 2025

PR approved by at least one committer and no changes requested.

@JNSimba JNSimba merged commit f2e75cb into apache:master Nov 3, 2025
27 of 29 checks passed
github-actions bot pushed a commit that referenced this pull request Nov 3, 2025
@JNSimba
[improve](job) Adding FailureReason after manually changing the statu…
0e72e4e 
…s of a streaming job (#57551)

### What problem does this PR solve?

Adding FailureReason after manually changing the status of a streaming
job
@github-actions github-actions bot mentioned this pull request Nov 3, 2025
Merged
yiguolei pushed a commit that referenced this pull request Nov 7, 2025
@github-actions @JNSimba
branch-4.0: [improve](job) Adding FailureReason after manually changi…
eaa7d32 
…ng the status of a streaming job #57551 (#57612)

Cherry-picked from #57551

Co-authored-by: wudi <wudi@selectdb.com>
@JNSimba JNSimba mentioned this pull request Jan 14, 2026
Merged
16 tasks
JNSimba added a commit that referenced this pull request Jan 14, 2026
@JNSimba
[Fix](Streamingjob) fix streaming job auto resume clear error (#59868)
7ccdbc1 
### What problem does this PR solve?

Related PR: #57551 #59784

1. Currently, errors are automatically cleaned up during autoresume.
Therefore, if a task continues to generate errors and autoresume, the
errors may not be detected at the job level.

2. Data quality errors are flagged when retrieving tasks.
github-actions bot pushed a commit that referenced this pull request Jan 14, 2026
@JNSimba
[Fix](Streamingjob) fix streaming job auto resume clear error (#59868)
f0aee0c 
### What problem does this PR solve?

Related PR: #57551 #59784

1. Currently, errors are automatically cleaned up during autoresume.
Therefore, if a task continues to generate errors and autoresume, the
errors may not be detected at the job level.

2. Data quality errors are flagged when retrieving tasks.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@liaoxin01 liaoxin01 liaoxin01 approved these changes

+1 more reviewer

@sollhui sollhui sollhui approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/4.0.2-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@JNSimba @hello-stephen @doris-robot @liaoxin01 @sollhui @yiguolei