Netty WebSocket decodes partial text frames

[slandelle]

From http://tools.ietf.org/html/rfc6455 5.6:

The "Payload data" is text data encoded as UTF-8. Note that a particular text frame might include a partial UTF-8 sequence; however, the whole message MUST contain valid UTF-8.

But Netty provider decodes every text frame data into UTF-8.
Looks like a bug to me. @jfarcand WDYT?

[slandelle]

If I'm right and TextFrame data should be decoded, then WebSocketTextListener.onFragment(String fragment, boolean last) is wrong. The only message that would make sense is BodyPart, so it's up to the user to decide what to do. We don't want to force useless byte copies.

[slandelle]

@oleksiys WDYT? It seems Grizzly works the same (seemingly wrong) way, see org.glassfish.grizzly.websockets.frametypes.TextFrameType

[slandelle]

I had some explanations from @rlubke. It seems that Grizzly pushes back invalid bytes so that they come with the next frame and they don't lose chars.

Still.

First, I really think onFragment(String) makes no sense. A fragment is a chunk, only the full payload can be a String.

Then, I don't think I can implement this in the Netty provider: I'm not an Oracle employer working on an Oracle project and I can't fork the Oracle licensed Encoder/Decoder code.

[slandelle]

@jroper Have you seen this one? Does Play uses WebSocketTextListener.onFragment(String fragment, boolean last)? I'd like some feedback from @jfarcand first but I'm very tempted to drop it.

[jroper]

No, Play doesn't use WebSocketTextListener at all - we don't provide any abstraction for WebSocket client support, Play users that need WebSocket clients will generally use AHC directly.

Netty lazily decodes text frames when you call getText - in our server support, we never call getText on partial fragments, rather, we aggregate the bytes into a buffer (up to a configurable limit), and then decode ourselves when we get the last fragment. This ensures we don't have this problem.

My advice would be, in the Netty implementation, insert the Netty WebSocketFrameAggregator into the pipeline between the websocket handler and your handler, then you'll never get a fragment.

[slandelle]

Thanks for your feedback.

Well, there's use cases for notifying fragments, like dumping binary fragments in a file, or parsing some JSON in streaming mode.

The problem is that I think I'd have a very hard time getting proper text fragment as Strings without losing chars. Grizzly was able to do it because they're Oracle, so they had no legal issue forking the JDK8 UTF-8 decoder.

And as I don't have a use case for it (my real focus is Gatling, not AHC), I won't even try. And I doubt I'll ever get a community PR for this.

I think I'll go with changing the API. I'll also split the Listeners in two: if your concern is to listen on fragments, you don't want the overhead of having AHC build the full byte array or String for you.

[slandelle]

Actually, I don't even see a use case for decoded String fragments: JSON parsers such as Jackson support streaming from bytes.

[slandelle]

No answer from anyone, so I'm the king of the world!

[jroper]

I for one welcome our new async-http-client overlord.