gRPC query extension#15982
Conversation
Report stats & close statement
Simple server + query test works
The gRPC shaded jar module causes issues when loaded into an IDE. This PR hides the module from Maven (and hence IDEs) by building it implicitly in the grpc-query module. It is a hack, but it works.
grpc protobufwriter
add empty response and GrpcResponseHandler tests
Allow ITs outside of the 'cases' directory Refactoring
Build the grpc-query extension archive Build a jar for the test Protobuf message Fixes to the extension path mechanism
* Refactor template into cluster directory * Add verify, setup scripts * Forbid snapshots from upstream repos * Basic grpc-query IT test
Retains support for anonymous (test) usage
| <parent> | ||
| <groupId>org.apache.druid</groupId> | ||
| <artifactId>druid</artifactId> | ||
| <version>30.0.0-SNAPSHOT</version> |
There was a problem hiding this comment.
| <version>30.0.0-SNAPSHOT</version> | |
| <version>31.0.0-SNAPSHOT</version> |
The master branch has been updated to 31.0.0-SNAPSHOT
There was a problem hiding this comment.
Let's fix it and then merge it
|
This pull request has been marked as stale due to 60 days of inactivity. |
| } | ||
|
|
||
| message QueryRequest { | ||
| QueryType queryType = 1; |
There was a problem hiding this comment.
@rishabh Could you make that QueryType is the last field in this proto to preserve the original order to avoid breaking changes and also give QueryType a default value of SQL.
fix proto to be backward compatible
…nto grpc-query-extension
|
@FrankChen021 could you please take a look at this PR, it has been open for a while |
|
@findingrish - can you rebase it on master so we can merge it? |
There was a problem hiding this comment.
what is this file for?
|
@findingrish - the test failures look genuine |
Revives #14024 and additionally supports,
This PR doesn't have the shaded module for packaging gRPC and Guava libraries since
grpc-querymodule uses the same Guava version as that of Druid.The response is gRPC-specific. It provides the result schema along with the results as a binary "blob". Results can be in CSV, JSON array lines or as an array of Protobuf objects. If using Protobuf, the corresponding class must be installed along with the gRPC query extension so it is available to the Broker at runtime.
Example usage
Sample request,
When using Protobuf response format, bundle up your Protobuf classes
into a jar file, and place that jar file in the
$DRUID_HOME/extensions/grpc-querydirectory.Specify the response Protobuf message name in the request.
Release note
This is a "contrib" extension. We don't seem to document such extensions in Druid itself. The README.md can serve as documentation instead.
This PR has: