From cc8ab2e253ea8eb945cd376a31a23c901fb1f644 Mon Sep 17 00:00:00 2001 From: Alban Crequy Date: Fri, 13 Apr 2018 16:42:43 +0200 Subject: [PATCH] doc: add developer guidelines It's not obvious how the TAP output is displayed and the meaning of the exit value. This documentation is meant to be for developers who want to add a new validation test. Signed-off-by: Alban Crequy --- docs/devel_guidelines.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) create mode 100644 docs/devel_guidelines.md diff --git a/docs/devel_guidelines.md b/docs/devel_guidelines.md new file mode 100644 index 00000000..1ed99b3d --- /dev/null +++ b/docs/devel_guidelines.md @@ -0,0 +1,17 @@ +# Developer guidelines + +## How to write validation tests + +Each validation test has a `.go` file in the `validation/` directory and can be compiled into a `.t` file and executed independently. + +### TAP output + +Each validation test prints TAP output. +So far, we have two kinds of validation tests and they print the TAP output differently: +* tests using `util.RuntimeInsideValidate`: they start the process `runtimetest` inside the container and `runtimetest` prints the TAP output. The test process itself must not output anything to avoid mixing its output with the TAP output. Each test can only call `util.RuntimeInsideValidate` one time because several TAP outputs cannot be concatenated. +* tests using `util.RuntimeOutsideValidate`: they create a container but without executing `runtimetest`. The test program itself must print the TAP output. + +### Exit status + +When the runtime fails a test, the TAP output indicates so with "not ok" but the exit status of test program normally remains 0. +A non-zero exit status indicates a problem in the test program rather than in the runtime.