Fix double utf8 encoding for payloads

[darrachequesne]

Following #80

Currently, a payload containing only strings (no binary) transmitted over polling transport gets utf8 encoded / decoded twice, one by the transport (the content type is set to text/plain; charset=UTF-8) and one here (encodePacket is called with utf8encode to true).

Fixes socketio/engine.io#315

[darrachequesne]

That case (string payload non utf8-encoded) should not happen.

[darrachequesne]

Breaking change!

[Andarist]

@darrachequesne
even if that's a breaking change it breaks wrong behaviour to the correct one, couldnt it be released as patch in v1, or something like @next? would like to benefit from this, but making own fork of the repo requires me to fork all 3 - socket.io-client uses engine.io-client which uses engine.io-parser

Also im wondering about this fragment of the source code.

We are using on the server go's client instead of node's and as its more low-level programming language it tends to think more in terms of bytes than in characters. So basically when meeting a char like ą it would expect this header length to be longer by 1, unfortunately its calculated via .length instead of counting bytes, so it crashes on server side.

However im not sure how original node.js client works and what it expects as this header length - therefore where should this be fixed? here in the parser? or there in server's golang implementation?

For now I get it work using heavy monkey-patching but would like to get rid of this asap

const originalEncode = parser.encodePacket
parser.encodePacket = (packet, supportsBinary, utf8encode, callback) => originalEncode(
	packet,
	supportsBinary,
	false,
	msg => callback(Object.defineProperty(
		{ toString: () => msg },
		'length',
		{ get: () => utf8.encode(msg).length }
	))
)

[darrachequesne]

@Andarist actually I'm not sure if it should be backported, since different versions of server/client would not be able to understand each other, would they?

Yes, it seems the parser expect that the payload is already utf8 decoded, I'm not sure if that's correct. Related: #87

[Andarist]

@darrachequesne
do u know how many versions were bugged? how many would be incompatible?

also i aint sure how the protocol should be working exactly, especially in context of various transports combinations - but my monkey patch mentioned before was applied just to encoding function, decoding seems to work just fine, but this PR changes both

Note: i think that the frame length should be counted in future releases of protocol in bytes than utf8 chars, it simply makes more sense for more backends.

[getnamo]

What updates are needed to enable socket.io 2.0 support for the cpp client? socketio/socket.io-client-cpp#173

[aleen42]

I don't think that double utf8 encoding for payloads is a problem especially when browsers do not encode payloads as expected like socket servers do not respond with Content-Type: text/plain; charset=UTF-8.