Support embedded newline characters in names?

[sunfishcode]

In #141 I created a test which attempted to test all the ASCII control characters in exported symbol names. All of them worked except 0x0a, the ASCII newline character. The spec interpreter gave this error when I tried it:

test/names.wast:50.11-50.14: unclosed text literal

What is the intended behavior here? I don't presently have an opinion here; I could see arguments for restricting the character set in some says, but I could also see arguments that it should be entirely unrestricted.

[rossberg]

There's no particular reason for the current behaviour, other than the same regexp character in the lexer being used to define comment syntax. :)

I don't have an overly strong opinion either, but for hygiene and for the sake of following standard practice, I'd lean towards disallowing any ASCII control characters in literals (this still allows UTF8).

[jfbastien]

Unicode has more control characters that ASCII :-)
Fun times can be had if we allow bidi in names!

IIRC @zygoloid was telling me that Unicode has a very well defined set of character literals that all languages should be using. As the editor of C++ he was appalled that C++ chose to ignore this and go its own special and nonsensical route.

[sunfishcode]

I guess you refer to this?

One issue is that WebAssembly does actually want a wider set than the set any high-level language will be using for identifiers, because WebAssembly aims to support compilers that need to be able to mangle names into something unrepresentable in source languages.

The document above also says: "Generally if the programming language has case-sensitive identifiers, then Normalization Form C is appropriate". However, Unicode Normalization Form C is non-trivial. It would be unfortunate if every WebAssembly tool has to know how to validate and normalize identifiers just to correctly do symbol lookups.

Another concern is homograph confusion. Since WebAssembly identifiers aren't user-facing anyway, would it make sense to restrict the character set and have frontends mangle as needed? They'll probably always have to do some mangling in any case.

Another is whether ES modules impose any constraints on this domain.

Thoughts?

[jcbeyler]

Since we have to do mangling anyway, I don't think it matters really what we allow since the front-end can just mangle it entirely and put whatever it likes as the internal representation. That is what I've already done in my parser to remove certain characters that LLVM did not like for example.

[rossberg]

With #143 merged, are people okay with closing this issue?

[jcbeyler]

I would vote yes

[sunfishcode]

Having filed this, I think we can close this. WebAssembly doesn't want to require engines to be in the business of interpreting character sets, so the simplest thing is for it to just support arbitrary uninterpreted byte strings. The mappings to JS and other languages can define the correspondence to Unicode as appropriate.