Allow forking task runners to terminate and pickup their tasks.#1521
Allow forking task runners to terminate and pickup their tasks.#1521drcrallen wants to merge 2 commits intoapache:masterfrom
Conversation
|
As an alternative, I could fork this off to its own task runner type. |
There was a problem hiding this comment.
I think this name should be more specific about what it actually is: cachedThreadPool? unboundedThreadPool?
There was a problem hiding this comment.
Its actually not needed in this PR anymore, I'll remove it. But it probably should have been cachedThreadPool
|
Is there anything that makes sure the RTR doesn't fail the tasks when the MM goes offline? Normally it does that as soon as the ephemeral task status announcements disappear, which would happen immediately. |
|
@gianm : no not yet |
drcrallen
force-pushed
the
forkingTaskRunnerRecoverable
branch
2 times, most recently
from
40bc654 to
5227553
Compare
drcrallen
force-pushed
the
forkingTaskRunnerRecoverable
branch
from
5227553 to
284273e
Compare
* Communication between MM and peon is accomplished via file system stuff. (adding and deleting files) * Force killing a peon is only available on systems which understand the `kill` command with -15 and -9 as options * Currently does not follow prior behavior. There is no option to force tasks to shutdown on MM exit * Adds ShutdownResource which adds an endpoint which shuts down the service (currently only used by peon) * Add unit tests for ForkingTaskRunner. These take a while to run because they spawn up JVMs so they are @ignore'd by default * Add tools.jar dependency for indexing-service for ForkingTaskRunner
drcrallen
force-pushed
the
forkingTaskRunnerRecoverable
branch
from
284273e to
8830f65
Compare
|
@gianm @drcrallen just following up with a status update here |
|
@fjy I have 1 TODO @ https://github.com/druid-io/druid/pull/1521/files#diff-d19c7676e29a15ee2e34062c8e4ee6eaR947 which I'm hoping to resolve today. Also I need more testing of Remote Task Runner when MM is offline to make sure it doesn't fail the task. I haven't put a version # on this PR and I still put the Discussion tag to try and help globally communicate it's status. |
|
I also haven't figured out a good solution for #1521 (comment) yet |
|
@drcrallen you may want to propose this in the forum of talk about it at the next dev sync |
|
@drcrallen or just ping @gianm as I'm sure he's thought about it |
There was a problem hiding this comment.
did you mean to call lifecycle.shutdown() somewhere here or register it as a shutdown hook. I don't see lifecycle being used in this class.
There was a problem hiding this comment.
I did at one point due to how peons were shutting down, but now it is all done through shutdown hooks and this will be removed
There was a problem hiding this comment.
ok, yeah, just saw the other PR with the update to CliPeon .
5 participants
killcommand with -15 and -9 as optionsThis allows upgrading the middle manager without killing peons.
Requires #1524