Conversation
Report stats & close statement
Simple server + query test works
The gRPC shaded jar module causes issues when loaded into an IDE. This PR hides the module from Maven (and hence IDEs) by building it implicitly in the grpc-query module. It is a hack, but it works.
grpc protobufwriter
add empty response and GrpcResponseHandler tests
Allow ITs outside of the 'cases' directory Refactoring
Build the grpc-query extension archive Build a jar for the test Protobuf message Fixes to the extension path mechanism
* Refactor template into cluster directory * Add verify, setup scripts * Forbid snapshots from upstream repos * Basic grpc-query IT test
Retains support for anonymous (test) usage
Remove the Python basic auth code: it is another PR
|
The build is failing due to Per the documentation:
This step does a compilation, but does not trigger creation of the shaded jar, which then causes compile errors. I've tried adding At this point, I won't be available to continue the fixes. Can someone else please continue chipping away at the various build issues? It seems the shaded jar does not play well with our build scripts: I've made several changes to work around the issue: more may be needed. Or, perhaps we're doing something wrong with the shaded jar. Thanks! |
Proposal apache#13469 Original PR apache#14024 A new method is being added in QueryLifecycle class to authorise a query based on authentication result. This method is required since we authenticate the query by intercepting it in the grpc extension and pass down the authentication result.
|
This pull request has been marked as stale due to 60 days of inactivity. |
|
This pull request/issue has been closed due to lack of activity. If you think that |
Revives #14024 and additionally supports, Native queries gRPC health check endpoint This PR doesn't have the shaded module for packaging gRPC and Guava libraries since grpc-query module uses the same Guava version as that of Druid. The response is gRPC-specific. It provides the result schema along with the results as a binary "blob". Results can be in CSV, JSON array lines or as an array of Protobuf objects. If using Protobuf, the corresponding class must be installed along with the gRPC query extension so it is available to the Broker at runtime.
2 participants
Implementation of a gRPC query endpoint per issue #13469.
Provides a single gRPC-based endpoint for SQL queries. The query request is similar to the existing REST
SqlQueryclass. The response is gRPC-specific. It provides the result schema along with the results as a binary "blob". Results can be in CSV, JSON array lines or as an array of Protobuf objects. If using Protobuf, the corresponding class must be installed along with the rRPC query extension so it is available to the Broker at runtime.The PR includes both unit and integration tests. The PR includes additional files that are also offered in other PRs. The intent is that those other PRs (#13877 and #14009) are merged first, then this one merges with master so that the other files "disappear" from this PR.
This PR also has a number of code-cleanup changes encountered while doing the implementation.
See the
README.mdfile in the PR for details. See thequery.protofile for the gRPC protocol and Protobuf messages.he project consists of three Maven modules:
grpc-query: The actual gRPC query endpoint.grpc-query-it: Integration tests for the extension.grpc-shaded: Creates a shaded jar containing gRPC and our rRPC service definition.The shaded module is needed because gRPC uses a version of Guava different than that which Druid uses. We get runtime errors if we try to combine the two. The shaded module contains the service definition because that generates code that also uses Guava.
The IT module has to be separate because it has dependencies that occur in the Maven build after the gRPC module. Specifically, it depends on it-cases which must come after distribution, but grpc-query must come before distribution.
Release note
This is a "contrib" extension. We don't seem to document such extensions in Druid itself. The
README.mdcan serve as documentation instead.This PR has: