Unexpected cancellation when subsequent message in client stream exceeds max size

[jhump]

This is almost a duplicate of #3996. That issue was ostensibly fixed in #11073. (And that same PR also seems to have fixed the issue reported in #10971.)

However, that fix seems to only work reliably when the error pertains to the first request message in a stream. So it works fine for unary and server-stream RPCs. However, for client-stream and bidi-stream RPCs, if some message in the request stream after the first one exceeds the configured max size, the old behavior still exists:

• A warning is logged with an exception that indicates the expected/correct error code:

  WARNING: Exception processing message
  io.grpc.StatusRuntimeException: RESOURCE_EXHAUSTED: gRPC message exceeds maximum size 204800: 204810
  	at io.grpc.Status.asRuntimeException(Status.java:525)
  	at io.grpc.internal.MessageDeframer.processHeader(MessageDeframer.java:392)
  	at io.grpc.internal.MessageDeframer.deliver(MessageDeframer.java:272)
  	at io.grpc.internal.MessageDeframer.request(MessageDeframer.java:162)
  	at io.grpc.internal.AbstractStream$TransportState$1RequestRunnable.run(AbstractStream.java:264)
  	at io.grpc.netty.NettyServerStream$TransportState$1.run(NettyServerStream.java:174)
  	at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
  	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)
  	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
  	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:569)
  	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
  	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
  	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
  	at java.base/java.lang.Thread.run(Thread.java:1583)
  

• But then a RST_STREAM frame is sent and the client instead observes a CANCELLED code.

[ejona86]

Yeah, the fix was known to only work in certain cases. Note that it doesn't matter how many requests the client sent. It should only matter whether the server has sent any responses (technically, whether the headers have been sent).

I mention the effort that'd be needed to more fully deliver better errors at #3996 (comment) . The current code sends the pretty error only if the headers haven't been sent. (The other conditions in the code aren't interesting here.)

grpc-java/netty/src/main/java/io/grpc/netty/NettyServerHandler.java

Line 821 in 781b4c4

return stream != null && !stream.isHeadersSent();