[DNM] dummy: clh: test PR #1099 from kata-containers/packaging

[likebreath]

Again this is a dummy PR to the test repo, and is used to validate the
changes to another PR from the packaging PR related to
cloud-hypervisor. The following linked issue is just a place holder.

Depends-on: github.com/kata-containers/packaging#1099
Depends-on: github.com/kata-containers/runtime#2900

Fixes: #2546

Signed-off-by: Bo Chen chen.bo@intel.com

[likebreath]

/test-clh

[likebreath]

/test-clh

[likebreath]

/test-clh

[likebreath]

/test-clh

[likebreath]

Interesting CI failure:

• clh-docker: It failed on [Fail] docker attach check attach functionality. It may be related to the 'potential regression failure on VmInfoGet failed' tracked here: clh: Potential regression test failure VmInfoGet failed runtime#2864. They all happened at the 'same' time frame.

• clh-metrics: failing at the very early setup stage as we also observed around the same time from another PR: clh: Add support to unplug block devices runtime#2833.

19:42:18 Report Summary:
19:42:18 +-----+----------------------+---------------------+---------------+------+-----+-----+-----+-----+-----+-----+
19:42:18 | P/F |         NAME         |         FLR         |     MEAN      | CEIL | GAP | MIN | MAX | RNG | COV | ITS |
19:42:18 +-----+----------------------+---------------------+---------------+------+-----+-----+-----+-----+-----+-----+
19:42:18 | *F* | boot-times           | Failed to load JSON | exit status 2 |      |     |     |     |     |     |     |
19:42:18 | *F* | memory-footprint     | Failed to load JSON | exit status 2 |      |     |     |     |     |     |     |
19:42:18 | *F* | memory-footprint-ksm | Failed to load JSON | exit status 2 |      |     |     |     |     |     |     |
19:42:18 | *F* | blogbench            | Failed to load JSON | exit status 2 |      |     |     |     |     |     |     |
19:42:18 | *F* | blogbench            | Failed to load JSON | exit status 2 |      |     |     |     |     |     |     |
19:42:18 +-----+----------------------+---------------------+---------------+------+-----+-----+-----+-----+-----+-----+

Restart both CI jobs:
/test-clh-docker
/test-clh-metrics

[likebreath]

The number is here. Surprisingly, the metrics CI is NOT failing on max memory footprint (10:32:27 time="2020-07-30T17:32:27Z" level=debug msg=" Check maxval (298119.885000 < 269135.350000)"), but it is failing on minimum memory footprint (10:32:27 time="2020-07-30T17:32:27Z" level=warning msg="Failed Minval ( 269727.515000(minimal mean expected) > 269135.350000(minimal mean on dataset)) for [memory-footprint]"). The same as we observed on the other PRs.

It looks like we will need to update the metrics CI so that it won't fail on minmum memory footprint (while keep reporting as a warning if we want this information). @amshinde @jcvenegas @GabyCT

[likebreath]

/test-clh

[likebreath]

/test-clh-metrics

[likebreath]

/test-clh

[likebreath]

/test-clh

[likebreath]

/test-clh-metrics

[likebreath]

Failed w/ 11:35:00 time="2020-08-14T18:35:00Z" level=warning msg="Failed Minval ( 269742.458500(minimal mean expected) > 269741.400000(minimal mean on dataset)) for [memory-footprint-ksm]". Log is here: http://jenkins.katacontainers.io/job/kata-metrics-tests-ubuntu-clh-18-04-PR/85/console

11:35:00 Report Summary:
11:35:00 +-----+----------------------+-------+--------+--------+-------+--------+--------+-------+------+-----+
11:35:00 | P/F |         NAME         |  FLR  |  MEAN  |  CEIL  |  GAP  |  MIN   |  MAX   |  RNG  | COV  | ITS |
11:35:00 +-----+----------------------+-------+--------+--------+-------+--------+--------+-------+------+-----+
11:35:00 | P   | boot-times           | 83.3% | 90.7%  | 116.7% | 33.3% | 82.3%  | 101.1% | 22.9% | 5.2% |  20 |
11:35:00 | P   | memory-footprint     | 95.0% | 95.0%  | 105.0% | 10.0% | 95.0%  | 95.0%  | 0.0%  | 0.0% |   1 |
11:35:00 | *F* | memory-footprint-ksm | 95.0% | 95.0%  | 105.0% | 10.0% | 95.0%  | 95.0%  | 0.0%  | 0.0% |   1 |
11:35:00 | P   | blogbench            | 80.0% | 112.1% | 120.0% | 40.0% | 112.1% | 112.1% | 0.0%  | 0.0% |   1 |
11:35:00 | P   | blogbench            | 80.0% | 105.4% | 120.0% | 40.0% | 105.4% | 105.4% | 0.0%  | 0.0% |   1 |
11:35:00 +-----+----------------------+-------+--------+--------+-------+--------+--------+-------+------+-----+

Restart the metrics CI as the difference of failed check is very small (e.g. 269742.458500(minimal mean expected) > 269741.400000(minimal mean on dataset)).
/test-clh-metrics

[likebreath]

/test-clh

[likebreath]

All CI jobs related to CLH are passing after the fix to the metrics CI, expect the fedora one (which is a known failure on the setup of docker/podman). I am restarting the CI jobs one more time again to test the packaging PR that is now rebased on its master branch.

[likebreath]

/test-clh

[likebreath]

/test-clh

[likebreath]

/test-clh

[likebreath]

All CI jobs are passing w/ seccomp option disabled. Restart the CIs to see how stable they are.
/test-clh

[likebreath]

For the second time, all CI jobs are passing w/ seccomp option disabled. The only exception is I needed to the CI job jenkins-ci-ubuntu-1804-cloud-hypervisor-minimal (log here) was failing on sudo -E PATH="$PATH" bash -c "make conformance", which passed after restarting the CI.

[likebreath]

Restarting the CI jobs on CLH the last/third time.
/test-clh

[likebreath]

All CI jobs related to cloud-hypervisor are passing 3 times in a roll w/ the static build PR, which proves the stability of using static build of clh. Closing the dummy test PR now.