-
Notifications
You must be signed in to change notification settings - Fork 3.7k
[enhance](job) delay load task schedule when transaction fail #57092
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
run buildall |
ClickBench: Total hot run time: 30.35 s |
FE UT Coverage ReportIncrement line coverage |
a8d6375 to
cf33165
Compare
|
run buildall |
FE Regression Coverage ReportIncrement line coverage |
dataroaring
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
|
PR approved by at least one committer and no changes requested. |
|
PR approved by anyone and no changes requested. |
dataroaring
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
liaoxin01
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Now, when transaction fail, routine load task will retry as soon as possible, when meet some temporarily unrecoverable errors like `too_many_version`, it will retry too many times in a short time and take huge pressure to upstream system like Kafka. To solve this problem, we delay load task schedule when transaction fail to reduce retry times when meet error and restore normal schedule if transactions resume normal execution.
Now, when transaction fail, routine load task will retry as soon as possible, when meet some temporarily unrecoverable errors like `too_many_version`, it will retry too many times in a short time and take huge pressure to upstream system like Kafka. To solve this problem, we delay load task schedule when transaction fail to reduce retry times when meet error and restore normal schedule if transactions resume normal execution.
…en transaction fail apache#57092 (apache#5207) cherry-picks from apache#57092 Related to apache#5195 Co-authored-by: hui lai <laihui@selectdb.com>
What problem does this PR solve?
Now, when transaction fail, routine load task will retry as soon as possible, when meet some temporarily unrecoverable errors like
too_many_version, it will retry too many times in a short time and take huge pressure to upstream system like Kafka.To solve this problem, we delay load task schedule when transaction fail to reduce retry times when meet error and restore normal schedule if transactions resume normal execution.
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)