Ambiguity with `COMPLETE` flag in `Response` frame

[NiteshKant]

Background

Response Frame is designed to eliminate the need of exchanging an additional frame for COMPLETE in cases where it is known while emitting an onNext that there are no more items.

This optimization is pretty useful for Request-Response exchange when it is known beforehand that there would only be one response frame (post fragmented frame coalescing).

In practice, this is less useful for request-stream, request-channel interactions where the presence of a functional composition library makes it more complex to determine whether an item is the last in the stream.

Problem

Since there is no explicit indication that a RESPONSE frame is NEXT_COMPLETE or just COMPLETE, a way to determine whether one should emit an item before completing is to check whether this RESPONSE contains any data or not. Java Implementation uses this approach.

As shown in this PR, this approach is error-prone for request-response interaction as an empty payload converts to just emit onComplete(). I will consider this as an error as request-response should expect either a response or an error.

[tmontgomery]

I had a discussion about this from the Reactive Streams API perspective. Unless the implementation does batching, there isn't a means to signal an onNext that also is an onComplete. At least that I can figure out.

[benjchristensen]

this approach is error-prone for request-response interaction as an empty payload

Why would there be an empty payload? In request-response, I agree that either "onNext+onComplete" or "onError" is required.

A request_response should always receive a NEXT_COMPLETE. It should be an error to even send a response frame without a payload on request_response.

Do we need to add something to the spec that clarifies or enforces this?

[NiteshKant]

@benjchristensen I think empty content is a valid usecase, eg: HTTP 204 is a valid status code which does not have any content. One option could be that we also check for metadata length but that could also be empty especially if it is over H2 which does header compression.

@tmontgomery Yes that is my belief too. Even if we do batching, there may be a function downstream that adds something to the batch. eg: stream.window().concat(). So, I don't think there is an effective way of doing this apart from the degenerate case of the Publisher being a special instance of a Publisher that holds a realized value eg: Flowable.just() in RxJava.

[benjchristensen]

But this isn't HTTP, and onComplete doesn't carry messages, so something like an "HTTP 204" either would have to be delivered in an onNext payload, or as an onError message.

So no, I don't think it's valid, just because HTTP does it. ReactiveSocket isn't HTTP.

[NiteshKant]

would have to be delivered in an onNext payload, or as an onError message.

So are you suggesting that when the payload is empty, we have two options:

• Send two frames: one ON_NEXT and one COMPLETE frame.
• Send ERROR that the payload is empty.

(Currently, we send a NEXT_COMPLETE frame (RESPONSE with complete flag set))

If yes, then which one do you prefer?

[benjchristensen]

There should not be an empty request-response that just completes. That breaks the ReactiveStreams spec which does not allow nulls. So if someone tries to return a null (an empty response), then it has to be onError.

[NiteshKant]

I think empty byte buffer and null are two different cases.

[benjchristensen]

Then send an empty byte buffer, as long as your mimetype knows what to do with it.

[benjchristensen]

The point is that request/response expects a non-null response to represent success. The ReactiveSocket state machine will always process the payload and emit onNext for request/response.

[benjchristensen]

Sounds like we should nail this down in the spec ... basically for each of the interaction models, what the expected contract is.

• request/response
• request/stream
• channel
• fire-and-forget

[stevegury]

I think an empty (!= null) response is perfectly valid, thus we need to distinguish between an empty onNext and an onComplete.

I agree that we should specify all of that in the spec.

[benjchristensen]

How would "empty" work with something like the Single (or a 'Future') type which is expected to work with request/response? It expects an onSuccess call, and will blow up with onError if onNext+onComplete are not called.

Are you saying an empty response would only call onComplete and not invoke onNext?

[stevegury]

In the case of request/response implemented with Single there's no difference, if it is implemented with an Observable, I expect that an empty response trigger onNext then onComplete, right now, it only trigger onComplete.

It is more pathologic in the case of request/stream, where an empty response will be interpreted as an onComplete and then end the stream from the client pov.

[benjchristensen]

request/stream is fine, as an empty stream with onComplete is part of the spec. It's just request/response, where Future/Single as a type expects onSuccess/onFailure methods.

So I don't see how "there's no difference" with Single, since Single will blow up and cause an onError if an onComplete is received without an onNext. It expects a single onNext followed by an onComplete.

[abersnaze]

@NiteshKant are going for a Maybe like request/response?