Add tests for if/then/else with indices + unknown label;

[bnjbvr]

Hi! I'd like to add test cases we ran into in our implementation and which could be useful for others:

• a test for if/then/else where the depths are not labels, but raw integer indices.
• a test for if/then/else where the label target of a br in the else is the label of the then block.

Unfortunately, the second test (at the end of the file) doesn't work: the parser complains that the label $l is unknown, which is exactly what we are testing here. The correct behavior would be to silently swallow the error and consider the test as passed. Am I missing something here?

[rossberg]

The current behaviour is correct: the body of each if-branch is (implicitly) a block, and the labels name these respective blocks. Not sure why you expect a label to be in scope in a sibling block? It's no different from

(if (...) (then (block $l ...)) (else (block (br $l))))

which likewise is an error.

The other test look good.

[kripken]

@rossberg-chromium I think it's not meant to be in scope, it should be an error - it's an assert_invalid test.

[rossberg]

Oh, sorry, totally missed that. Yeah, that's definitely a bug, I'll look into it.

[rossberg]

Okay, I now see what's going on. The problem is that labels (and identifiers in general) are resolved in the parser, because the AST doesn't have them. An unbound name hence is a syntax error, rejecting the script before you ever get to run it.

So it's not really a bug but a limitation of scripts -- a script can't check for this kind of error, just like it can't check other syntax errors. You'll need to check externally against expected-output instead.

[binji]

@rossberg-chromium I actually ran an issue in sexpr-wasm because of this; prior to having binary modules, I could assume that assert_invalid would only be given modules that were syntactically valid. So I moved the checking pass out of the parser, and the parser always returns a module as an AST, even inside assert_invalid. But now that binary modules can fail to parse, I need a second path for passing back binary data that is only later parsed and checked for errors.

[bnjbvr]

So it's not really a bug but a limitation of scripts -- a script can't check for this kind of error, just like it can't check other syntax errors. You'll need to check externally against expected-output instead.

Thank you for the explanation, I can do this, however I wonder how https://github.com/WebAssembly/spec/blob/master/ml-proto/test/switch.wast#L155 doesn't trigger the exact same issue?

[rossberg]

@bnjbvr, that test uses a raw numeric index. What's resolved in the parser is the use of symbolic labels, and their translation into numeric ones (because they are purely syntactic sugar, not a part of the Wasm AST).

I certainly won't argue that this difference is nice. :)

[bnjbvr]

I've updated the PR. The code wasn't accepting non-zero exit codes, so I had to refactor this, by adding a way to specify that a test is expected to fail. Also, there were issues with redirecting stderr to stdout with a > in the command argument, so I've refactored the way output is written so that stdout and stderr goes to the output file, instead. How does this look?

[rossberg]

Instead of instrumenting comments in the file, what do you think about simply using a file name convention? Say, <test>.fail.wast for tests that ought to fail? That's easier and cheaper to process (including for other test harnesses).

[bnjbvr]

Yeah, good idea.

[rossberg]

Thanks! Looks good modulo a few comments.

[bnjbvr]

Updated the latest patch, thank you for the review!

[rossberg]

Can we make this a bit narrower? ".fail." should be good.

[rossberg]

Hm, Travis still barfs at the .fail test, although I can't immediately say why.

[rossberg]

One further style nit I just noted: inconsistent indentation, rest of the file uses 2

[bnjbvr]

Right, the test was failing because we are now very tied to line numbers, which is unfortunate. Addressed the two styles nits and updated the expected output to match line numbers.

[rossberg]

Cool, thanks! LGTM and landing.