Skip to content

Specify that tests must be output in order #10

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
iHiD merged 3 commits into from
Jan 6, 2021
Merged

Specify that tests must be output in order #10

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
6ba0aba
Specify that tests must be run in order
iHiD Jan 5, 2021
57dd0e6
Update anatomy/track-tooling/test-runners/interface.md
ErikSchierboom Jan 6, 2021
e604ea7
Update anatomy/track-tooling/test-runners/interface.md
iHiD Jan 6, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions anatomy/track-tooling/test-runners/interface.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,17 @@ Where the status is `error` (the tests fail to execute cleanly), the top level `

When the status is not `error`, either set the value to `null` or omit the key entirely.

#### Tests

> key: `tests`

This is an array of the test results, specified in the "Per-test" section below.

The tests **MUST** be returned in the order they are specified in the tests file.
For languages that execute tests in a random order, this may mean re-ordering the results in line with the order specified in the tests file.
Comment thread
ErikSchierboom marked this conversation as resolved.

The rationale for this is that only the first failure is shown to students and therefore it is important that the correct failure is shown. Because tests are generally ordered in the tests file in a TDD way, and because for Practice Exercises the students see the tests file in the editor, aligning the results with the tests file is critical.

### Per-test

#### Name
Expand Down