diff --git a/anatomy/track-tooling/test-runners/interface.md b/anatomy/track-tooling/test-runners/interface.md index aa5c69fd..dfceaf55 100644 --- a/anatomy/track-tooling/test-runners/interface.md +++ b/anatomy/track-tooling/test-runners/interface.md @@ -53,6 +53,17 @@ Where the status is `error` (the tests fail to execute cleanly), the top level ` When the status is not `error`, either set the value to `null` or omit the key entirely. +#### Tests + +> key: `tests` + +This is an array of the test results, specified in the "Per-test" section below. + +The tests **MUST** be returned in the order they are specified in the tests file. +For languages that execute tests in a random order, this may mean re-ordering the results in line with the order specified in the tests file. + +The rationale for this is that only the first failure is shown to students and therefore it is important that the correct failure is shown. Because tests are generally ordered in the tests file in a TDD way, and because for Practice Exercises the students see the tests file in the editor, aligning the results with the tests file is critical. + ### Per-test #### Name