'shutdownAllTasks' API for a dataSource

[QiuMM]

Try to fix #6115

[gianm]

This namespace collides with {taskId}. It's not likely that anyone will actually create a task with the same name as a datasource, but you never know, so let's use a different URL here. How about: /druid/indexer/v1/dataSource/{dataSource}/shutdownAllTasks

[gianm]

IMO, we should return OK in this case, because that makes the kill-all operation idempotent. It's a good property to have, especially in something that might be done "just in case" when removing a datasource.

[QiuMM]

If so, I'll remove the ownTask variable and return Response.ok(ImmutableMap.of("dataSource", dataSource)).build() for all case. Is that OK?

[gianm]

That sounds good to me… thanks!

[QiuMM]

fix!

[surekhasaharan]

You can also use this API getActiveTaskInfo(@Nullable String dataSource) which takes a dataSource as argument and filters at the DB query time, so you will only get the matching dataSource active tasks. So like below. It might be efficient as you don't have to compare every task's dataSource in the for loop.

final List<TaskInfo<Task, TaskStatus>> tasks = taskStorageQueryAdapter.getActiveTaskInfo(dataSource);
            for (final TaskInfo<Task, TaskStatus> task : tasks) {
              taskQueue.shutdown(task.getId());
            }``` 

[QiuMM]

Cool, fixed!

[asdf2014]

Maybe tasks.parallelStream().forEach(t -> taskQueue.shutdown(t.getId())) will be better.

[QiuMM]

I do not think so, taskQueue.shutdown is a synchronized operation. Use parallelStream has no performance improvement.

[asdf2014]

Yes, you are right. Even if ReentrantLock is good at handling with concurrent scenes, we should not try to use parallelStream here. I didn't notice it. My fault. Forget it, what do you think of converting it into a more functional programming style, such as tasks.forEach(t -> taskQueue.shutdown(t.getId()))?

[QiuMM]

I think there is no need. Other for loop code in class OverlordResource are not functional programming style. Coding style should keep consistency.

[asdf2014]

Okay, it's up to you. 😅

[gianm]

Agree that it's good to be consistent within a file, and the rest of this file is totally down with for loops.

[QiuMM]

@gianm Can you help restart the failed travis job. Thanks.

[nishantmonu51]

have restarted the failing travis.