[Backport] MSQ profile for Brokers and Historicals. (#17140) (#17072)

[kfaraz]

This patch adds a profile of MSQ named "Dart" that runs on Brokers and Historicals, and which is compatible with the standard SQL query API. For more high-level description, and notes on future work, refer to #17139.

This patch contains the following changes, grouped into packages.

Controller (org.apache.druid.msq.dart.controller):

The controller runs on Brokers. Main classes are,

• DartSqlResource, which serves /druid/v2/sql/dart/.
• DartSqlEngine and DartQueryMaker, the entry points from SQL that actually run the MSQ controller code.
• DartControllerContext, which configures the MSQ controller.
• DartMessageRelays, which sets up relays (see "message relays" below) to read messages from workers' DartControllerClients.
• DartTableInputSpecSlicer, which assigns work based on a TimelineServerView.

Worker (org.apache.druid.msq.dart.worker)

The worker runs on Historicals. Main classes are,

• DartWorkerResource, which supplies the regular MSQ WorkerResource, plus Dart-specific APIs.
• DartWorkerRunner, which runs MSQ worker code.
• DartWorkerContext, which configures the MSQ worker.
• DartProcessingBuffersProvider, which provides processing buffers from sliced-up merge buffers.
• DartDataSegmentProvider, which provides segments from the Historical's local cache.

Message relays (org.apache.druid.messages):

To avoid the need for Historicals to contact Brokers during a query, which would create opportunities for queries to get stuck, all connections are opened from Broker to Historical. This is made possible by a message relay system, where the relay server (worker) has an outbox of messages.

The relay client (controller) connects to the outbox and retrieves messages. Code for this system lives in the "server" package to keep it separate from the MSQ extension and make it easier to maintain. The worker-to-controller ControllerClient is implemented using message relays.

Other changes:

• Controller: Added the method "hasWorker". Used by the ControllerMessageListener to notify the appropriate controllers when a worker fails.
• WorkerResource: No longer tries to respond more than once in the "httpGetChannelData" API. This comes up when a response due to resolved future is ready at about the same time as a timeout occurs.
• MSQTaskQueryMaker: Refactor to separate out some useful functions for reuse in DartQueryMaker.
• SqlEngine: Add "queryContext" to "resultTypeForSelect" and "resultTypeForInsert". This allows the DartSqlEngine to modify result format based on whether a "fullReport" context parameter is set.
• LimitedOutputStream: New utility class. Used when in "fullReport" mode.
• TimelineServerView: Add getDruidServerMetadata as a performance optimization.
• CliHistorical: Add SegmentWrangler, so it can query inline data, lookups, etc.
• ServiceLocation: Add "fromUri" method, relocating some code from ServiceClientImpl.
• FixedServiceLocator: New locator for a fixed set of service locations. Useful for URI locations.

Fixes #XXXX.

Description

Fixed the bug ...

Renamed the class ...

Added a forbidden-apis entry ...

Release note

---

Key changed/added classes in this PR

• MyFoo
• OurBar
• TheirBaz

---

This PR has:

• been self-reviewed.
  • using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)
• added documentation for new or modified features or behaviors.
• a release note entry in the PR description.
• added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
• added or updated version, license, or notice information in licenses.yaml
• added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
• added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
• added integration tests.
• been tested in a test Druid cluster.