Requests may have non-sequential stream ids

[mostroverkhov]

Happens when request (for all interactions) subscriptions race

rsocket-java/rsocket-core/src/main/java/io/rsocket/RSocketRequester.java

Line 218 in c66dfb8

RequestResponseFrameFlyweight.encode(

- so frames are sent not in same order as stream ids are created

rsocket-java/rsocket-core/src/main/java/io/rsocket/RSocketRequester.java

Line 209 in c66dfb8

int streamId = streamIdSupplier.nextStreamId(receivers);

. Other than being against spec this makes implementing http2 streams transport harder that necessary - http2 request stream ids are strictly sequential also as I cant trust rsocket stream ids. Just for discussion - will have reproducer by the end of the week.

[OlegDokuka]

Other than being against spec this makes implementing http2 streams

Can you please point to a certain spec section that says anything on that?

request stream ids are strictly sequential also as I cant trust rsocket stream ids. Just for discussion - will have reproducer by the end of the week.

I can not see any possible solution without involving complex synchronization in order to ensure the streams ID are identical to the order in which payloads come into the network. Even though the id is generated at the same time the payload, there is always racing on delivering a frame.

Also, it makes sense for HTTP/2 (https://tools.ietf.org/html/rfc7540#section-5.1.1) which is based on TCP connection but now imagine Aeron or QUIC connection
which does not guarantee anything in terms of the order of frames so having such a restriction just going to complicate implementation unreasonably.

I guess the reference to HTTP/2 is more on the fashion how ids are generated rather than on that particular restriction related to the fact that HTTP/2 is TCP based

What do you think on that @nebhale @smaldini @rstoyanchev @rdegnan @linux-china @stevegury @yschimke?

[mostroverkhov]

Can you please point to a certain spec section that says anything on that?

I read that as they should be sequential

Stream IDs on the client MUST start at 1 and increment by 2 sequentially, such as 1, 3, 5, 7, etc.
Stream IDs on the server MUST start at 2 and increment by 2 sequentially, such as 2, 4, 6, 8, etc.

which is based on TCP connection but now imagine Aeron or QUIC connection
which does not guarantee anything in terms of the order of frames

Transports are required to send all frames in order. Given above, if there are 2 client requests, there
should be two frames on wire with ids 1,3, both sent and received in that order.

I can not see any possible solution without involving complex synchronization

One solution from top of my head - existing (and future ones, like quic) transport implementations are netty based so there is EventLoop that can be used as scheduler for serialization. This should have negligible performance impact - done by (reactor)netty anyways, just pull this task up from transport to rsocket.

[OlegDokuka]

@mostroverkhov the implementation does not contradict anything said in the spec. StreamIdSupplier issues ids sequentially.

One solution from top of my head - existing (and future ones, like quic) transport implementations are netty based so there is EventLoop that can be used as scheduler for serialization. This should have negligible performance impact - done by (reactor)netty anyways, just pull this task up from transport to rsocket.

yeah, that works out

---

A QUIC stream provides reliable in-order delivery of bytes, but makes no guarantees about order of delivery with regard to bytes on other streams.

Due to HTTP/3 spec, messages within a logical stream are delivered in order but there are no guarantees of delivering messages in total order see (https://quicwg.org/base-drafts/draft-ietf-quic-http.html#section-6).

It means that issuing stream id 1, flushing it to the network, issuing stream id 3 flushing it to the network may end up delivering stream id 3 earlier than stream id 1, which is not a big deal. I guess the most important is that packets within a logical stream are delivered with the order of packets issuing.

Maybe we have to clarify spec with regards to upcoming network protocols.

What do you think @nebhale @linux-china @smaldini @rdegnan @yschimke @tmontgomery @rstoyanchev @spencergibb