[C++][FlightRPC] Decouple Arrow Flight RPC from GRPC

[asfimport]

I'm trying to implement Flight RPC on RPC framework with protobuf message support in distributed system.

However, the flight rpc is tied to grpc.
Classes from grpc used in flight server are:

1. grpc::ServerContext used in grpc generated code in parameter, and used to generate ServerCallContext.
2. grpc::Status used in grpc generated code as return type.
3. grpc::ServerReaderWriter and grpc::ServerReader used in massive wrapped MessageReader/Writer classes.

1 & 2 are not coupled much with flight, while the third part is the tough work.
Shall we introduce an interface class with same semantics to allow anyone implement the writing process to stream, such as arrow::flight::ServerReaderWriter and arrow::flight::ServerReader.

So that, making a shim layer between FlightServiceImpl and FlightServerBase is possible to decouple flight from grpc, meanwhile taking advantage of its zero-copy messages.
All message converting processes can be handled in the shim layer.
For example, the function definition of DoGet can be arrow::Status DoGet(ServerCallContext\* context, const pb::Ticket\* request, ServerWriter<pb::FlightData>\* writer), which converts pb messages to flight's and call functions from actual business logic implementation from FlightServerBase as Status DoGet(const ServerCallContext& context, const Ticket& request, std::unique_ptr<FlightDataStream>\* stream).

While, the client seems more complex, since the cookie stuff and others.
If the idea above is possible, I'll have a exploration on client in depth

Reporter: Wenbo Hu

Related issues:

• [C++][FlightRPC] Support non-grpc data planes (duplicates)

_{Note: This issue was originally created as ARROW-13889. Please see the migration documentation for further details.}

[asfimport]

David Li / @lidavidm:
Hey Wenbo, you may want to kick this up to the dev mailing list dev@arrow.apache.org for more visibility. We have wanted to support alternative transports in Flight but so far haven't looked at how best to do that. What transport are you looking at? One thought was to keep gRPC for all control plane methods (GetFlightInfo, DoAction, etc.) and support alternative transports for data plane methods (DoGet, DoPut, DoExchange).

Having a shim over the gRPC specific classes sounds like a reasonable approach. Frankly, what's probably needed is a lot of refactoring of the build scripts and code organization so that different transports can be toggled on and off. And possibly, reorganization such that different transports can be implemented as contrib modules, depending on what transports exactly you'd like to support. But this is something for discussion and I would encourage you to talk to the mailing list!