Ability to encode additonal DATA during encodeTrailers

[lizan]

In gRPC transcoding, we translate streamed gRPC message to JSON objects like:

[<JSON translated grpc-message>
,<JSON translated grpc-message>
,<JSON translated grpc-message>
]

Is there anyway to encode additional DATA frame in encodeTrailers, while encodeData(data, false) returned Continue? We need this to encode the ] at the end.

cc #501

[mattklein123]

@lizan @fengli79 let me think about this. I will get back to you tomorrow.

[fengli79]

+1, for grpc-web, I also need to encode the upstream http2 trailers in the downstream http1 response body.
There's a StreamEncoderFilterCallbacks, with a buffer inside. It will be nice if we can use that buffer to carry the data translated from trailers.

[mattklein123]

@lizan @fengli79 this is in encoder filter case, right? So per @fengli79 if I make it so that if you populate encodingBuffer() from inside encodeTrailers(), that buffer gets flushed first, I assume that will solve your problem?

[fengli79]

@mattklein123 Yes for my case.

[lizan]

@mattklein123 Yes, that would work. I tried that approach in current envoy, it didn't work (crashed).

[mattklein123]

I don't think modifying the decoding/encoding buffer inline is going to work here without some super confusing behavior where we keep track of what was already in the buffer and then only dispatch the delta. This is because we could have already been buffering, so just checking the existence of the buffer and dispatching all of it doesn't make sense. I'm thinking through this to figure out if this is the right approach or if we should just add additional callback functions.

[mattklein123]

@lizan @fengli79 I think I have a solution, but I'm going to change the filter API a little bit to remove write access to decoding/encodingBuffer and add an explicit addBodyData() function. This will require a slight code change on your part but I think will make all of this a lot simpler to grok. I will figure this out locally and then put up a PR for you both to look at.