Add feature for MSQ to output payload in CSV (And other formats) currently supported by Druid SQL API

[acherla]

Issue: Druid MSQ API today does not support different output formats similar to Druid SQL API for the MSQ engine. Adding support for CSV,JSON and Protobuff formats would make it easier to export data to downstream systems.

Druid Version: 25.0.0

Why is this important: Allowing different types of formats should reduce payload size for a front facing application or for exporting data from MSQ to a remote system.

[cryptoe]

Thanks @acherla for raising this. Certainly this seems useful. As we think through how to expose this functionality, it would be helpful to get more details about your use cases:

• How big is your druid cluster. ( a super rough estimate is also fine)
• What's the size of data you want to export.
• Do you have a need to fetch the results multiple times.
• How will you consume the exported results. ( Serially, in parallel)
• What kind of summary of the data you are looking from druid in your result set.
• How will the next system discover the schema of this exported result.

[acherla]

1. Druid cluster today is 500+ nodes
2. Around 100K to 500K records
3. No, these reports will be generated using the MSQ task engine since they are high latency queries
4. Serially
5. No need for summary, should reflect the output of the MSQ query if its a select statement
6. via the header (CSV format) or JSON object key/value fields

[github-actions]

This issue has been marked as stale due to 280 days of inactivity.
It will be closed in 4 weeks if no further activity occurs. If this issue is still
relevant, please simply write any comment. Even if closed, you can still revive the
issue at any time or discuss it on the dev@druid.apache.org list.
Thank you for your contributions.

[github-actions]

This issue has been closed due to lack of activity. If you think that
is incorrect, or the issue requires additional review, you can revive the issue at
any time.

[cryptoe]

Done as part of druid 29.0.0 :
#15689