21-1 Whitelist Nightly Test Failure: Cannot Connect to Docker daemon

[sgairo]

2/1 Nightly VIC Engine Test failure:
Suites.Group21-Registries.21-1-Whitelist.zip

Cannot connect to the Docker daemon at tcp://sc-rdops-vm01-dhcp-30-126.eng.vmware.com:2376. Is the docker daemon running?

[sgairo]

Feeling this is a fluke, will investigate further to confirm.

[sgairo]

Potentially seen in 2/6 nightly 6.0 test run. No logs, but console output says every test failed as "Harbor doesn't come up properly!"

[sgairo]

This is essentially a slow test, worth seeing ways to speed it up, otherwise will close.

[zjs]

Historical Data

* [PASS] VIC Engine Nightly Build - 6.5 - 2018-02-15-00-15-26 
* [PASS] VIC Engine Nightly Build - 6.0 - 2018-02-14-20-32-00 
* [PASS] VIC Engine Nightly Build - 6.5 - 2018-02-14-00-48-22 
* [PASS] VIC Engine Nightly Build - 6.0 - 2018-02-13-20-32-00 
* [FAIL] VIC Engine Nightly Build - 6.5 - 2018-02-13-00-52-33
   * No logs for this suite included in overall results .tar.gz, but all three tests failed with "Harbor failed to come up properly!"
   * The tests failed due to a suite setup failure. No details are available because the failing keyword, `Install Harbor To Test Server`, is tagged secret.
* [PASS] VIC Engine Nightly Build - 6.0 - 2018-02-12-20-32-00 
* [PASS] VIC Engine Nightly Build - 6.5 - 2018-02-12-00-41-35 
* [PASS] VIC Engine Nightly Build - 6.0 - 2018-02-11-20-32-00 
* [PASS] VIC Engine Nightly Build - 6.5 - 2018-02-11-00-10-15 
* [PASS] VIC Engine Nightly Build - 6.0 - 2018-02-10-20-32-00 
* [PASS] VIC Engine Nightly Build - 6.5 - 2018-02-09-23-21-06 
* [PASS] VIC Engine Nightly Build - 6.0 - 2018-02-09-20-32-00 
* [PASS] VIC Engine Nightly Build - 6.5 - 2018-02-09-00-18-27 
* [PASS] VIC Engine Nightly Build - 6.0 - 2018-02-08-20-32-00 
* [PASS] VIC Engine Nightly Build - 6.5 - 2018-02-07-23-52-33 
* [PASS] VIC Engine Nightly Build - 6.0 - 2018-02-07-20-32-00 
* [PASS] VIC Engine Nightly Build - 6.5 - 2018-02-07-00-38-14 
* [FAIL] VIC Engine Nightly Build - 6.0 - 2018-02-06-20-32-00
   * No logs for this suite included in overall results .tar.gz, but all three tests failed with "Harbor failed to come up properly!"
   * The tests failed due to a suite setup failure. No details are available because the failing keyword, `Install Harbor To Test Server`, is tagged secret.
* [PASS] VIC Engine Nightly Build - 6.5 - 2018-02-05-23-41-55 
* [PASS] VIC Engine Nightly Build - 6.0 - 2018-02-05-20-32-00 
* [PASS] VIC Engine Nightly Build - 6.5 - 2018-02-05-00-47-00 
* [PASS] VIC Engine Nightly Build - 6.0 - 2018-02-04-20-32-00 
* [PASS] VIC Engine Nightly Build - 6.5 - 2018-02-04-00-10-51 
* [PASS] VIC Engine Nightly Build - 6.0 - 2018-02-03-20-32-00 
* [PASS] VIC Engine Nightly Build - 6.5 - 2018-02-03-00-25-02 
* [PASS] VIC Engine Nightly Build - 6.0 - 2018-02-02-20-32-00 
* [PASS] VIC Engine Nightly Build - 6.5 - 2018-02-01-20-22-04 
* [FAIL] VIC Engine Nightly Build - 6.0 - 2018-02-01-12-52-23
   * The Configure Registry CA test failed with a "context deadline exceeded" while attempting to perform a `vic-machine-linux configure` with the default timeout value (3m). (Logs linked in original report.)

Analysis

Since the test started intermittently failing on February 1, it has failed in 3 out of 28 runs (10.7%).

There appear to be two distinct ways in which it has failed:

1. "context deadline exceeded" while attempting to perform a vic-machine-linux configure
2. suite setup failure

Possible next steps

1. Ensure that tests in the suite set a larger timeout when invoking vic-machine commands. It seems like other tests set timeouts of 15 or 30 minutes.
2. Determine if the Install Harbor To Test Server keyword can be refactored to provide more useful output when a failure occurs (e.g., by more narrowly scoping the secret tagging).