Skip to content

Conversation

@lygstate
Copy link
Contributor

@lygstate lygstate commented Jan 15, 2021

When running tests262 cases, the jerry program may exit with error code other than 1,
that's means a crash or JERRY_ASSERT triggered and should be resolved, so dump the test
name and stderr/stdout for tracking those crash in CI.

Depends on #4492

JerryScript-DCO-1.0-Signed-off-by: Yonggang Luo luoyonggang@gmail.com

@ossy-szeged
Copy link
Contributor

I like the idea to track crashes, assertions, timeouts, etc. But I would be much stricter.
In my opinion crashes and assertions isn't acceptable at all, an engine shouldn't do it on any input.
There is only one exception: OOM is the expected results for TCO tests, since jerry isn't support tail call optimization.

But unfortunately there are crashes and assertions now, so first we should fix them, then we can make
the harness to interpret any assertion and crash as failure (in case of negative test too).

I'm not against to have more verbose errors for these tests in test262.report file. But we should filter
them to have compact output of run-test --test262. Maybe we can extend the summary to count
assertions and crashes, and the user can check the test262.report to investigate the details.

@lygstate lygstate force-pushed the logging-crash branch 4 times, most recently from bab8c6f to 126dbd7 Compare January 18, 2021 08:29
When running tests262 cases, the jerry program may exit with error code other than 1,
that's means a crash or JERRY_ASSERT triggered and should be resolved, so dump the test
name and stderr/stdout for tracking those crash in CI.

related issue: jerryscript-project#4463

JerryScript-DCO-1.0-Signed-off-by: Yonggang Luo luoyonggang@gmail.com
Copy link
Member

@dbatyai dbatyai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@rerobika rerobika merged commit dd698f8 into jerryscript-project:master Feb 4, 2021
@lygstate lygstate deleted the logging-crash branch February 4, 2021 14:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants