-
Notifications
You must be signed in to change notification settings - Fork 3.7k
[fix](job) remove can not transform RUNNING to NEED_SCHEDULE limit #52887
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
run buildall |
TPC-H: Total hot run time: 33453 ms |
TPC-DS: Total hot run time: 186621 ms |
ClickBench: Total hot run time: 29.2 s |
liaoxin01
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
|
PR approved by at least one committer and no changes requested. |
|
PR approved by anyone and no changes requested. |
dataroaring
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
…52887) ### What problem does this PR solve? Routine load job could not transform RUNNING to NEED_SCHEDULE, when partition num increase and reschedule job, it will throw exception, causing new partition can not consume: ``` 2025-07-07 14:35:39,847 WARN (Routine load scheduler|41) [RoutineLoadScheduler.runAfterCatalogReady():59] Failed to process one round of RoutineLoadScheduler org.apache.doris.common.DdlException: errCode = 2, detailMessage = Could not transform RUNNING to NEED_SCHEDULE at org.apache.doris.load.routineload.RoutineLoadJob.checkStateTransform(RoutineLoadJob.java:788) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.load.routineload.RoutineLoadJob.unprotectUpdateState(RoutineLoadJob.java:1366) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.load.routineload.RoutineLoadJob.update(RoutineLoadJob.java:1483) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.load.routineload.RoutineLoadManager.updateRoutineLoadJob(RoutineLoadManager.java:839) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.load.routineload.RoutineLoadScheduler.process(RoutineLoadScheduler.java:65) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.load.routineload.RoutineLoadScheduler.runAfterCatalogReady(RoutineLoadScheduler.java:57) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.common.util.MasterDaemon.runOneCycle(MasterDaemon.java:58) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.common.util.Daemon.run(Daemon.java:116) ~[doris-fe.jar:1.2-SNAPSHOT] ``` introduced by #40728, and should remove this limit.
…52887) ### What problem does this PR solve? Routine load job could not transform RUNNING to NEED_SCHEDULE, when partition num increase and reschedule job, it will throw exception, causing new partition can not consume: ``` 2025-07-07 14:35:39,847 WARN (Routine load scheduler|41) [RoutineLoadScheduler.runAfterCatalogReady():59] Failed to process one round of RoutineLoadScheduler org.apache.doris.common.DdlException: errCode = 2, detailMessage = Could not transform RUNNING to NEED_SCHEDULE at org.apache.doris.load.routineload.RoutineLoadJob.checkStateTransform(RoutineLoadJob.java:788) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.load.routineload.RoutineLoadJob.unprotectUpdateState(RoutineLoadJob.java:1366) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.load.routineload.RoutineLoadJob.update(RoutineLoadJob.java:1483) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.load.routineload.RoutineLoadManager.updateRoutineLoadJob(RoutineLoadManager.java:839) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.load.routineload.RoutineLoadScheduler.process(RoutineLoadScheduler.java:65) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.load.routineload.RoutineLoadScheduler.runAfterCatalogReady(RoutineLoadScheduler.java:57) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.common.util.MasterDaemon.runOneCycle(MasterDaemon.java:58) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.common.util.Daemon.run(Daemon.java:116) ~[doris-fe.jar:1.2-SNAPSHOT] ``` introduced by #40728, and should remove this limit.
…52887) ### What problem does this PR solve? Routine load job could not transform RUNNING to NEED_SCHEDULE, when partition num increase and reschedule job, it will throw exception, causing new partition can not consume: ``` 2025-07-07 14:35:39,847 WARN (Routine load scheduler|41) [RoutineLoadScheduler.runAfterCatalogReady():59] Failed to process one round of RoutineLoadScheduler org.apache.doris.common.DdlException: errCode = 2, detailMessage = Could not transform RUNNING to NEED_SCHEDULE at org.apache.doris.load.routineload.RoutineLoadJob.checkStateTransform(RoutineLoadJob.java:788) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.load.routineload.RoutineLoadJob.unprotectUpdateState(RoutineLoadJob.java:1366) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.load.routineload.RoutineLoadJob.update(RoutineLoadJob.java:1483) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.load.routineload.RoutineLoadManager.updateRoutineLoadJob(RoutineLoadManager.java:839) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.load.routineload.RoutineLoadScheduler.process(RoutineLoadScheduler.java:65) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.load.routineload.RoutineLoadScheduler.runAfterCatalogReady(RoutineLoadScheduler.java:57) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.common.util.MasterDaemon.runOneCycle(MasterDaemon.java:58) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.common.util.Daemon.run(Daemon.java:116) ~[doris-fe.jar:1.2-SNAPSHOT] ``` introduced by #40728, and should remove this limit.
What problem does this PR solve?
Routine load job could not transform RUNNING to NEED_SCHEDULE, when partition num increase and reschedule job, it will throw exception, causing new partition can not consume:
introduced by #40728, and should remove this limit.
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)