please respect unicode strings you are given

[vkz]

esprima.parse('a = "привет"') clearly preserves unicode strings in the format supplied

{ type: 'Program',
  body: 
   [ { type: 'ExpressionStatement',
       expression: 
        { type: 'AssignmentExpression',
          operator: '=',
          left: { type: 'Identifier', name: 'a' },
          right: { type: 'Literal', value: 'привет', raw: '"привет"' } } } ] }

esgen(esprima.parse('a = "привет"')) clearly does not

a = '\u043F\u0440\u0438\u0432\u0435\u0442';

Technically these js-strings are equivalent, but why forego readability?

[michaelficarra]

Use verbatim: "raw" in your escodegen options if you want the literal representation preserved.

[vkz]

thank you!

Does fix the unicode representation, but wraps the expression in a redundant set of parens:
a = ('привет');

PS I have to ask though, why mangle the string literal in the first place?

[michaelficarra]

It's not getting mangled. Those are the same strings. We only have the raw representation and the value. We can render the value using only Unicode escapes if we want, but we can't print all characters in a string literal safely. So I think we only print non-control ASCII characters and escape all others.

[vkz]

can't say I follow. What does render safely mean? Escodegen received both raw and value from the outside world, so someone somewhere both rendered and read them safely. AST is fair game, but I don't understand why it would need to mess with the value. (if you squint a little you might even say that escodegen is breaking its own type-signature by messing with values instead of AST nodes).

More generally here's where I'm coming from. I think escodegen is often used in projects that are required to consume and produce human-readable code, which, seeing how the world writes in more than one language, may on occasion include some unicode. Present behaviour is unexpected to say the least, because it presents itself to a customer as something completely baffling. At the minimum it seems like the wrong default.

[fpirsch]

👍

[michaelficarra]

@vkz

What does render safely mean?

We can't have " in "-delimited strings. We can't have a carriage return or a paragraph separator. To avoid these characters, we can just Unicode escape everything. It's the same string. If you really want to avoid Unicode escapes, use {format: {escapeless: true}} as your escodegen options. But the only way to use the same escapes that you used in your original code is to use the {verbatim: "raw"} option.

[vkz]

right. Chance of us agreeing on the matter seems to be dwindling with each message, so let's not prolong the agony. If I may I'll leave you with these observations:

You already have the string representation in hand and it's clearly not broken, why fear then? Not saying you're wrong especially with security in the mix, but it is the wrong default.

But the only way to use the same escapes that you used in your original code

I'm using Russian not escapes just like other people could've used Japanese, Hindi, etc.

[michaelficarra]

But you very well could have used escapes. See the following program for example.

("\0 \x00 \u0000 0 \x30 \u0030");

There are many ways I could have escaped that string but I chose that way. Preserving this requires the verbatim option. Are you getting it now?

[vkz]

var esprima = require("esprima"),
    esgen = require("escodegen").generate,
    uglify = require("uglify-js");

var exhibit1 = '"\0 \x00 \u0000 0 \x30 \u0030"',
    ast1 = esprima.parse(exhibit1),
    exhibit2 = "'привет'",
    ast2 = esprima.parse(exhibit2);

esgen(ast1);
uglify.AST_Node.from_mozilla_ast(ast1).print_to_string();

esgen(ast2);
uglify.AST_Node.from_mozilla_ast(ast2).print_to_string();

every time uglify-js does this horribly unsafe thing a puppy dies.

Having lost several hours of my life debugging this safe behaviour I feel some friendly trolling is warranted here. There'll be many a poor sobs down the road tripping over this, I'm sure.

NB: Esprima and Escodegen have saved me countless hours, I use them daily and I am absolutely grateful to their authors and maintainers for keeping them open source and providing support.