Allow Dart to query realtime tasks#18076
Conversation
gianm
left a comment
There was a problem hiding this comment.
Generally looking good. I don't see an IT - did you test it to make sure it works?
Btw, I don't think we have Dart integration tests yet, so I won't ask you to add an IT in this patch, but we should address that in the future.
Yes, I have tested it locally, with only real-time segments as well as a mix. I too noticed that there isn't a Dart IT. I will add one in another PR. |
| .stream() | ||
| .filter(druidServerMetadata -> includeSegmentSource.getUsedServerTypes() | ||
| .contains(druidServerMetadata.getType())) | ||
| .filter(druidServerMetadata -> SegmentSource.REALTIME.getUsedServerTypes() |
There was a problem hiding this comment.
To mirror the suggestion for IndexerControllerContext, consider moving this to a constant in DartControllerContext.
There was a problem hiding this comment.
Changed! This one instance should probably remain as REALTIME, not a default value. This part of the code is common to both and should only be called if realtime segments are queried.
Add the ability for Dart to query real-time tasks.
Most of the changes are around including the realtime segments while slicing the inputs in the controller. There are also some small changes around handling worker counts in the case of no historical segments.
Additionally,
cloneQueryModeis now respected by Dart.Release Notes
includeSegmentSourcetorealtime, in a similar way to MSQ tasks. Dart queries real-time tasks by default.This PR has: