MSQ: Fix task lock checking during publish, fix lock priority.#13282
Merged
cryptoe merged 7 commits intoapache:masterfrom Nov 8, 2022
Merged
MSQ: Fix task lock checking during publish, fix lock priority.#13282cryptoe merged 7 commits intoapache:masterfrom
cryptoe merged 7 commits intoapache:masterfrom
Conversation
Fixes two issues: 1) ControllerImpl did not properly check the return value of SegmentTransactionalInsertAction when doing a REPLACE. This could cause it to not realize that its locks were preempted. 2) Task lock priority was the default of 0. It should be the higher batch default of 50. The low priority made it possible for MSQ tasks to be preempted by compaction tasks, which is not desired.
gianm
added
Bug
Area - MSQ
cryptoe
approved these changes
Oct 29, 2022
Contributor
cryptoe
left a comment
cryptoe
left a comment
There was a problem hiding this comment.
Minor nit: LGTM otherwise.
| @Override | ||
| public int getPriority() | ||
| { | ||
| return getContextValue(Tasks.PRIORITY_KEY, Tasks.DEFAULT_BATCH_INDEX_TASK_PRIORITY); |
Contributor
There was a problem hiding this comment.
Nit: should we document this here https://druid.apache.org/docs/24.0.0/ingestion/tasks.html#lock-priority ?
Contributor
|
LGTM thanks @gianm 🚀 |
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes two issues:
ControllerImpl did not properly check the return value of
SegmentTransactionalInsertAction when doing a REPLACE. This could cause
it to not realize that its locks were preempted.
Task lock priority was the default of 0. It should be the higher
batch default of 50. The low priority made it possible for MSQ tasks
to be preempted by compaction tasks, which is not desired.