-
Notifications
You must be signed in to change notification settings - Fork 8
Open
Description
I've run into a problem where a UTF-8 encoded string is parsed by Simdjson.parse and one of the resulting strings is encoded in ASCII-8BIT. I can reproduce this like so:
# run with ruby --encoding=UTF-8 if UTF-8 isn't your system default.
x = '{"m":" – "}' # note the non-ASCII character in the value
puts x.encoding # => #<Encoding::UTF-8>
y = Simdjson.parse(x)
puts y['m'].encoding # => #<Encoding:ASCII-8BIT>It seems like the encoding of the output strings should remain the same as the encoding of the input string, right? I'm not sure if this is an issue that belongs here or in the main simdjson repository but I appreciate you taking a look either way.
simoleone
Metadata
Metadata
Assignees
Labels
No labels