Discussion of proposal 2015-006 (Additional string conversion functionality)

[rossberg-old]

This issue is for discussion of proposal 2015-006 (Additional string conversion functionality).

[JohnReppy]

I think that the REs for specifying the accepted formats are not the right ones. I propose

StringCvt.BIN   [+~-]?[0-1]+(_[0-1]+)*
StringCvt.OCT   [+~-]?[0-7]+(_[0-7]+)*
StringCvt.DEC   [+~-]?[0-9]+(_[0-9]+)*
StringCvt.HEX   [+~-]?(0x|0X)?[0-9a-fA-F]+(_[0-9a-fA-F]+)*

In other words, allow the underscore as a digit separator, but not as a initial or terminal character
in a numeric literal.

BTW, this change agrees with the way that I wrote up this feature in the Proposed Definition of SuccessorML.

[rossberg]

Good point. Shouldn't the underscore at least be allowed after a 0x, though?

[JohnReppy]

Something like 0x_5 looks weird to me, but perhaps it should be allowed. I just took a look at what Rust allows for numeric literals and it is

[+~-]?[0-9][_0-9]*

for decimal literals and

[+~-]?(0x|0X)?[0-9a-fA-F_]+

for hexadecimal literals (modulo the SML tilde as minus). Are there other examples of languages that use underscore in numeric literals that we might compare to?

[rossberg]

So 0x_ is a legal literal in Rust? That looks weird.

Ocaml has also introduced them a while ago. It does not seem to allow an underscore after 0x.

Whatever we pick, the rules for word prefixes should be consistent with it.

[JohnReppy]

The reference states

A hex literal starts with the character sequence U+0030 U+0078 (0x) and continues as any mixture of hex digits and underscores.

but the implementation may do something different.

[JohnReppy]

Java only allows underscores between digits, but allows an arbitrary number of them. see here.

[olopierpa]

On Tue, Aug 18, 2015 at 8:54 AM, John Reppy notifications@github.com
wrote:

Are there other examples of languages that use underscore in numeric
literals that we might compare to?

Ada (not at start of digits, not at end of digits, no two consecutive
underscores)

[larsbergstrom]

@rossberg 0x_ in Rust generates an error no valid digits found for number. See: https://play.rust-lang.org/?gist=ac5c4c71e296bed79a0e&version=stable

I think the text in the Rust language reference could possibly be more clear on this point: https://doc.rust-lang.org/stable/reference.html#integer-literals
CC @nmatsakis

[JohnReppy]

So it looks like the Rust language should be

A hex literal starts with the character sequence U+0030 U+0078 (0x) and continues as any mixture of hex digits and underscores containing at least one digit.

[MatthewFluet]

In the MLton implementation (https://github.com/MLton/mlton/blob/master/mlton/front-end/ml.lex#L152), we require leading and trailing digits and no consecutive underscores. As I recall, there is no ambiguity or lexical trouble allowing leading or trailing or repeated underscores, although it has some very minor impacts on backwards compatibility:

fun foo _1__2_ = true
   | foo _ _ _ _ _ _ = false

[JohnReppy]

This discussion should is better attached to the Successor ML Definition; the Basis Library should follow whatever we agree to do there. I've created an issue for that purpose.

[JohnReppy]

Closed by mistake.