Fix data race in getting results from MSQ select tasks.#16107
Merged
cryptoe merged 3 commits intoapache:masterfrom Mar 13, 2024
Merged
Fix data race in getting results from MSQ select tasks.#16107cryptoe merged 3 commits intoapache:masterfrom
cryptoe merged 3 commits intoapache:masterfrom
Conversation
github-actions
Bot
added
Area - Batch Ingestion
Area - MSQ
| resultsYielder.close(); | ||
| } | ||
| catch (IOException e) { | ||
| throw new RuntimeException("Unable to fetch results of various worker tasks successfully", e); |
Contributor
There was a problem hiding this comment.
nit: should be DruidException
Contributor
Author
There was a problem hiding this comment.
I can probably refactor this part of a another PR.
cryptoe
added a commit
to cryptoe/druid
that referenced
this pull request
Mar 18, 2024
* Fix data race in getting results from MSQ select tasks. * Add better logging * Handling number overflow.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With #15880 going in, it exposed a potential data race where the MSQ controller would post finish to the workers but still fetch the results from them.
The change pushes the result fetching before we issue a worker finish.
As a result of this change, now we would materialize all the rows of the select result in the controller memory. This is okay when
selectDestination=durableStoragebut a little problematic whenselectDestination=taskReportsince the former has a limit of 3000 rows but the later can go wild.The other way would be to hold of finishing the tasks till we write out the task report.