Should responses with content-type application/json have UTF-8 encoding by default?

[drwl]

Currently, responses without a specified charset encoding in the header are stored internally with binary/ascii 8bit encoding. According to the spec, the specified charset should be redundant. Should the default for json responses by utf-8 instead?

From RFC8259:

JSON text exchanged between systems that are not part of a closed
ecosystem MUST be encoded using UTF-8 [RFC3629].

Previous specifications of JSON have not required the use of UTF-8
when transmitting JSON text. However, the vast majority of JSON-
based software implementations have chosen to use the UTF-8 encoding,
to the extent that it is the only encoding that achieves
interoperability.

Also in RFC8259, at the end of Section 11 (or just above of Section 12):

Note: No "charset" parameter is defined for this registration.
Adding one really has no effect on compliant recipients.

[tarcieri]

That seems like a reasonable default, yes

[drwl]

That seems like a reasonable default, yes

I can try to put up a PR. Where do you suggest this code change take place?

My initial thought is to change HTTP::Response initializer:

        encoding   = opts[:encoding] || charset

        if encoding.nil?
          if mime_type == 'application/json'
            encoding = Encoding::UTF_8
          else
            encoding = Encoding::BINARY
          end
        end

thoughts?

[tarcieri]

Sounds good to me /cc @ixti

[ixti]

@tarcieri I like the idea. Reviewed the PR.

[drwl]

@ixti @tarcieri thanks for quick turn around. Not to rush, but do yall have any idea when I can expect a release with this change?

I came across this related to work and so I'm figuring out if I should make changes to support this internally or if I can wait and update the gem (when it's out).

[ixti]

Released as 5.1.0

[ixti]

I can cut release layer today

…

On Thu, 16 Jun 2022, 19:45 Andrew W. Lee, ***@***.***> wrote: @ixti <https://github.com/ixti> @tarcieri <https://github.com/tarcieri> thanks for quick turn around. Not to rush, but do yall have any idea when I can expect a release with this change? I came across this related to work and so I'm figuring out if I should make changes to support this internally or if I can wait and update the gem (when it's out). — Reply to this email directly, view it on GitHub <#714 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAXEPVV24CMIAFOIQS2ESLVPNR2DANCNFSM5YV3IZWQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>