[spec] Clarify that strings can separate tokens

[tlively]

Since " is not included in idchar, it can mark the end of a token. For
example, 0"foo"$x would parse as 0, "foo", $x.

[rossberg]

Oh, interesting. That is kind of unintentional and inconsistent (e.g., I remember that we explicitly decided to make 0$a a single token and tweaked the lexical grammar for that purpose at some point in the past).

I wonder if we can afford fixing it? That is, add " to the characters that can occur in reserved? Technically, that would be a breaking change to the text format, but it's probably unlikely to affect real-world code. (Edit: as a quick check I just tried, at least the test suite does not notice the difference.)

[tlively]

Yeah, I can see that it might be surprising that " starts a new token but $ does not. I wonder how many parsers in the wild actually implement this lexing correctly anyway. It's probably worth asking more broadly whether anyone cares before making a breaking change, but I also suspect this wouldn't matter in practice.

[rossberg]

Yes, absolutely. Shall we put it on the next meeting's agenda?

How did you run into this in the first place?

[tlively]

Sure, I’ll make an agenda PR. I found this because I’m working on a new spec-compliant text parser for Binaryen, so I’m taking a closer look at the text spec for the first time.

[tlively]

We had unanimous consent in the CG meeting this morning to make a change to the grammar to disallow strings from separating tokens, although we didn't get into the details of how such a change might be accomplished.

[rossberg]

See #1499.

[rossberg]

Closing this, since it has been superseded by a spec change.