[WIP] tests/int: run bats tests in parallel

[kolyshkin]

Currently this includes and is based on both #2741 and #2757.

Won't work: see #2759 (comment)

[kolyshkin]

So, runc requires a terminal (in case terminal: true) is set in config, and bats -j N (where N > 1, i.e. when GNU parallel is used) does not provide one:

$ cat b.bats 
@test 1 {
	stty size
}

@test 2 {
	stty size
}
$ bats b.bats 
 ✓ 1 
 ✓ 2 

2 tests, 0 failures

$ bats -j 2 b.bats 
 ✗ 1
   (in test file b.bats, line 2)
     `stty size' failed
   stty: 'standard input': Inappropriate ioctl for device
 ✗ 2
   (in test file b.bats, line 6)
     `stty size' failed
   stty: 'standard input': Inappropriate ioctl for device

2 tests, 2 failures

Before #2535 runc probably fails if under under bats -j, but after this commit it opens /dev/tty (as long as runc is not disowned, I guess), and sets some options for it.

Now we are doomed, because setting terminal options from a background process leads to SIGTTOU for the whole group. Here's a simple demo without runc:

$ cat c.bats 
@test 1 {
	stty size
}

@test 2 {
	stty size
}

@test 3 {
	stty -F /dev/tty sane
}
$ bats c.bats 
 ✓ 1 
 ✓ 2 
 ✓ 3 

3 tests, 0 failures

$ bats -j 3  c.bats 
# (hangs forever)

In another terminal we see

1195926 pts/5    S+     0:00  |       |   \_ /usr/bin/bash /usr/libexec/bats-core/bats -j 3 c.bats
1195933 pts/5    S+     0:00  |       |       \_ /usr/bin/bash /usr/libexec/bats-core/bats-exec-suite -j 3 -x /home/kir/go/src/github.com/opencontainers/runc/c.bats
1195944 pts/5    S+     0:00  |       |       |   \_ /usr/bin/perl /usr/bin/parallel --keep-order --jobs 3 bats-exec-file -j 3 -x {} /tmp/bats-run-1195926/test_list_file.txt
1195950 pts/5    S      0:00  |       |       |       \_ /usr/bin/bash /usr/libexec/bats-core/bats-exec-file -j 3 -x /home/kir/go/src/github.com/opencontainers/runc/c.bats /tmp/bats-run-1195926/t
1195960 pts/5    S      0:00  |       |       |           \_ /usr/bin/perl /usr/bin/parallel --semaphore -j 3 --fg -- bats-exec-test -x /home/kir/go/src/github.com/opencontainers/runc/c.bats test
1195999 pts/5    T      0:00  |       |       |               \_ /usr/bin/bash /usr/libexec/bats-core/bats-exec-test -x /home/kir/go/src/github.com/opencontainers/runc/c.bats test_3 3
1196007 pts/5    T      0:00  |       |       |                   \_ stty -F /dev/tty sane
1195934 pts/5    S+     0:00  |       |       \_ /usr/bin/bash /usr/libexec/bats-core/bats -j 3 c.bats
1195935 pts/5    S+     0:00  |       |       \_ /usr/bin/bash /usr/libexec/bats-core/bats-format-pretty

So, unless we fake a terminal (using script, ssh -tt, screen, tmux, expect or the like), we can't parallelize runc integration tests using bats -j.

[kolyshkin]

Closing for now (see the comment above for reasons why). Might need to address this in bats.