Skip to content

Cherry-pick: Changes to allow us to run nightly tests for 1.4.3#8151

Merged
zjs merged 31 commits intovmware:releases/1.4.3from
zjs:cherry/nightly-fixes
Jul 27, 2018
Merged

Cherry-pick: Changes to allow us to run nightly tests for 1.4.3#8151
zjs merged 31 commits intovmware:releases/1.4.3from
zjs:cherry/nightly-fixes

Conversation

@zjs
Copy link
Member

@zjs zjs commented Jul 19, 2018

Cherry-pick a variety of changes related to nightly testing to allow us to run nightly tests on the 1.4.3 release branch:

@zjs zjs self-assigned this Jul 19, 2018
@zjs zjs requested review from DanielXiao and hickeng July 19, 2018 08:41
hickeng
hickeng approved these changes Jul 26, 2018
View reviewed changes
Copy link
Contributor

@hickeng hickeng left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Running this via the test base job passed with a limited set of tests. For me that's enough to determine that the core of the changes are correct. We may need some specific test cherry picks as well, but this is the necessary core.

@zjs
Copy link
Member Author

zjs commented Jul 27, 2018

Before this is rebased and merged, it should be rebased to include this PR number on the first line of each commit message.

mhagen-vmware and others added 23 commits July 26, 2018 18:05
@mhagen-vmware @zjs
Update DRS test to our new non-drs support (vmware#7765) (vmware#8151)
7d8eedd 
(cherry picked from commit 453200e)
@zjs
Add retry on vSAN DOM objects in DS check (vmware#7792) (vmware#8151)
e13795c 
* Add retry on vSAN DOM objects in DS check

(cherry picked from commit f1c9787)
@zjs
Add vSAN DOM check retry for vSAN Complex test (vmware#7799) (vmware#…
87f782e 
…8151)

* Add vSAN DOM check retry for vSAN complex

(cherry picked from commit 760046b)
@zjs
Add nightly for vicadmin tests on isolated vch (vmware#7793) (vmware#…
c7ff9b3 
…8151)

Adds a nightly test that verifies vicadmin functions properly without a
wan connection. Fixes vmware#7102.

(cherry picked from commit d5135e4)
@jzt @zjs
Fix setup/teardown in opsuser tests (vmware#7836) (vmware#8151)
22662a5 
(cherry picked from commit c6ea3e2)
@zjs
nightly: Disable 5-25-OPS-User-Grant upgrade test (vmware#7863) (vmwa…
26a741a 
…re#8151)

Disable the "granted ops-user perms work after upgrade" test in the
5-25-OPS-User-Grant nightly suite.

(cherry picked from commit 7b6cde7)
@rajanashok @zjs
Nimbus testbed for nfs volume nightly test (vmware#7853) (vmware#8151)
41140c9 
Updated the NFS Volume nightlies to build a full NFS test bed instead of relying on the unsupported pod placement calls.

Fixes vmware#7642

(cherry picked from commit ff6baa9)
@jzt @zjs
Use 1.3.1 in ops user upgrade test (vmware#8151)
7e7a351 
(cherry picked from commit 47851e3)
@mhagen-vmware @zjs
Fix several tests that did not appropriately use the vic-admin env var (
3057da1 
vmware#7879) (vmware#8151)

(cherry picked from commit b5bad94)
@mhagen-vmware @zjs
Update the nightlies builds (vmware#7883) (vmware#8151)
b8c0173 
(cherry picked from commit 70e4c2b)
@zjs
Configure ROBO nightlies with Group19-DRS-Disabled (vmware#7876) (vmw…
f30e517 
…are#8151)

This commit adds test scripts for running a nightly Jenkins job to
run tests in a non-DRS environment. This change runs the
Group19-DRS-Disabled suite against a pre-deployed, static vCenter
ELM testbed. The corresponding secrets file will be committed to
the internal repo.

Future changes will include adding more tests and using a testbed
deployed with a service account.

Fixes vmware#7662

(cherry picked from commit fe040e0)
@mhagen-vmware @zjs
Add timeouts in case of hung operations (vmware#7884) (vmware#8151)
b463224 
(cherry picked from commit 72391e5)
@mhagen-vmware @zjs
New testbed for multiple cluster test (vmware#7888) (vmware#8151)
246796f 
(cherry picked from commit 0ce740b)
@mhagen-vmware @zjs
Switch to testbed for NFS Datastore test (vmware#7900) (vmware#8151)
d887fc9 
(cherry picked from commit 29f07bb)
@zjs
VCH install timeout set to 30m for 5-21-Datastore-Path (vmware#7902) (v…
0cd4c2f 
…mware#8151)

Increases timeout from default 3m to 30m

Toward vmware#7718

(cherry picked from commit 0e43ea6)
@mhagen-vmware @zjs
Add NFS datastore keyword back (vmware#7906) (vmware#8151)
9099872 
(cherry picked from commit b7784b8)
@mhagen-vmware @zjs
Select correct datastore for the test (vmware#7909) (vmware#8151)
b904d1b 
(cherry picked from commit 86c0c52)
@mhagen-vmware @zjs
Create a new testbed that includes a static IP address worker (vmware…
4dfcaa6 
…#7918) (vmware#8151)

(cherry picked from commit de44136)
@andrewtchin @zjs
Add ping to NFS Volume nightly tests (vmware#7821) (vmware#8151)
e9836e2 
(cherry picked from commit 71664a5)
@mhagen-vmware @zjs
Fix VSAN testbed for 6.7 testing (vmware#7971) (vmware#8151)
41df355 
(cherry picked from commit 4573c60)
@singhshwetaQE @zjs
Improving 21-1-Whitelist Test -Harbor Setup (vmware#7997) (vmware#8151)
a50da2d 
* fix 21-1-Whitelist test

(cherry picked from commit ef61c13)
@singhshwetaQE @zjs
fix harbor-util (vmware#8011) (vmware#8151)
450ec92 
(cherry picked from commit 2c5ded5)
@hickeng @zjs
Parameterize nightly job script (vmware#8047) (vmware#8151)
4d141d1 
This parameterizes the nightly job runner script so that we can pass in
the configuration elements that were previously hard coded. The primary
difference is it's possible to run a user specified subset of the test
cases.

Also removes references in the test definitions to specific user
directories.

There is a mostly duplicated version of the job runner files called
jenkins-drs-disabled-run.sh - as best I can determine these files are not
unified only because some of the nightly tests have not been confirmed to
work when the test executor is in a container.

(cherry picked from commit 3b269fa)
zjs pushed a commit that referenced this pull request Jul 27, 2018
@mhagen-vmware @zjs
Update DRS test to our new non-drs support (#7765) (#8151)
27991fe 
(cherry picked from commit 453200e)
zjs pushed a commit that referenced this pull request Jul 27, 2018
@zjs
Add retry on vSAN DOM objects in DS check (#7792) (#8151)
bbb1272 
* Add retry on vSAN DOM objects in DS check

(cherry picked from commit f1c9787)
zjs pushed a commit that referenced this pull request Jul 27, 2018
@zjs
Add vSAN DOM check retry for vSAN Complex test (#7799) (#8151)
7b6ad73 
* Add vSAN DOM check retry for vSAN complex

(cherry picked from commit 760046b)
zjs pushed a commit that referenced this pull request Jul 27, 2018
@zjs
Add nightly for vicadmin tests on isolated vch (#7793) (#8151)
2aebb23 
Adds a nightly test that verifies vicadmin functions properly without a
wan connection. Fixes #7102.

(cherry picked from commit d5135e4)
zjs pushed a commit that referenced this pull request Jul 27, 2018
@jzt @zjs
Fix setup/teardown in opsuser tests (#7836) (#8151)
595b8ab 
(cherry picked from commit c6ea3e2)
zjs added a commit that referenced this pull request Jul 27, 2018
@zjs
nightly: Disable 5-25-OPS-User-Grant upgrade test (#7863) (#8151)
f698990 
Disable the "granted ops-user perms work after upgrade" test in the
5-25-OPS-User-Grant nightly suite.

(cherry picked from commit 7b6cde7)
zjs pushed a commit that referenced this pull request Jul 27, 2018
@rajanashok @zjs
Nimbus testbed for nfs volume nightly test (#7853) (#8151)
3c41feb 
Updated the NFS Volume nightlies to build a full NFS test bed instead of relying on the unsupported pod placement calls.

Fixes #7642

(cherry picked from commit ff6baa9)
zjs pushed a commit that referenced this pull request Jul 27, 2018
@jzt @zjs
Use 1.3.1 in ops user upgrade test (#8151)
fd3f32d 
(cherry picked from commit 47851e3)
zjs pushed a commit that referenced this pull request Jul 27, 2018
@mhagen-vmware @zjs
Fix several tests that did not appropriately use the vic-admin env var (
79ca33e 
#7879) (#8151)

(cherry picked from commit b5bad94)
zjs pushed a commit that referenced this pull request Jul 27, 2018
@mhagen-vmware @zjs
Update the nightlies builds (#7883) (#8151)
62ca620 
(cherry picked from commit 70e4c2b)
zjs pushed a commit that referenced this pull request Jul 27, 2018
@zjs
Configure ROBO nightlies with Group19-DRS-Disabled (#7876) (#8151)
98b6f07 
This commit adds test scripts for running a nightly Jenkins job to
run tests in a non-DRS environment. This change runs the
Group19-DRS-Disabled suite against a pre-deployed, static vCenter
ELM testbed. The corresponding secrets file will be committed to
the internal repo.

Future changes will include adding more tests and using a testbed
deployed with a service account.

Fixes #7662

(cherry picked from commit fe040e0)
zjs pushed a commit that referenced this pull request Jul 27, 2018
@mhagen-vmware @zjs
Add timeouts in case of hung operations (#7884) (#8151)
e0b70c9 
(cherry picked from commit 72391e5)
zjs pushed a commit that referenced this pull request Jul 27, 2018
@zjs
Fix tar extraction path in DRS-disabled nightlies (#7889) (#8151)
66442c8 
This change fixes the VIC tar extraction path in the
jenkins-drs-disabled-run.sh script. We were erroneously
extracting the binaries outside the vic/ directory, causing the
'Install VIC Appliance To Test Server' keyword to fail because it
couldn't locate the vic-machine binary in vic/bin.

Towards #7662

(cherry picked from commit 0cf0634)
zjs pushed a commit that referenced this pull request Jul 27, 2018
@zjs
Add test for no-DRS VCH create with RP options (#7881) (#8151)
080b072 
This commit adds an integration test in Group19-DRS-Disabled to
verify that during a VCH create, vic-machine checks for the DRS
setting in the targeted cluster and also warns if any command
options specific to resource pools are supplied, since they will
be ignored as they are not applicable in a DRS-disabled env.

Towards #7275

(cherry picked from commit e3991c1)
zjs pushed a commit that referenced this pull request Jul 27, 2018
@mhagen-vmware @zjs
New testbed for multiple cluster test (#7888) (#8151)
31eb789 
(cherry picked from commit 0ce740b)
zjs pushed a commit that referenced this pull request Jul 27, 2018
@mhagen-vmware @zjs
Switch to testbed for NFS Datastore test (#7900) (#8151)
7244a5f 
(cherry picked from commit 29f07bb)
zjs pushed a commit that referenced this pull request Jul 27, 2018
@zjs
VCH install timeout set to 30m for 5-21-Datastore-Path (#7902) (#8151)
8871e8f 
Increases timeout from default 3m to 30m

Toward #7718

(cherry picked from commit 0e43ea6)
zjs pushed a commit that referenced this pull request Jul 27, 2018
@mhagen-vmware @zjs
Add NFS datastore keyword back (#7906) (#8151)
e2341ad 
(cherry picked from commit b7784b8)
zjs pushed a commit that referenced this pull request Jul 27, 2018
@mhagen-vmware @zjs
Select correct datastore for the test (#7909) (#8151)
84063de 
(cherry picked from commit 86c0c52)
zjs pushed a commit that referenced this pull request Jul 27, 2018
@mhagen-vmware @zjs
Create a new testbed that includes a static IP address worker (#7918) (…
a58b478 
…#8151)

(cherry picked from commit de44136)
zjs pushed a commit that referenced this pull request Jul 27, 2018
@andrewtchin @zjs
Add ping to NFS Volume nightly tests (#7821) (#8151)
38eb5bd 
(cherry picked from commit 71664a5)
zjs pushed a commit that referenced this pull request Jul 27, 2018
@mhagen-vmware @zjs
Fix VSAN testbed for 6.7 testing (#7971) (#8151)
6c8c418 
(cherry picked from commit 4573c60)
zjs pushed a commit that referenced this pull request Jul 27, 2018
@singhshwetaQE @zjs
Improving 21-1-Whitelist Test -Harbor Setup (#7997) (#8151)
7fdff99 
* fix 21-1-Whitelist test

(cherry picked from commit ef61c13)
zjs pushed a commit that referenced this pull request Jul 27, 2018
@singhshwetaQE @zjs
fix harbor-util (#8011) (#8151)
4669702 
(cherry picked from commit 2c5ded5)
zjs pushed a commit that referenced this pull request Jul 27, 2018
@hickeng @zjs
Parameterize nightly job script (#8047) (#8151)
b53a4bb 
This parameterizes the nightly job runner script so that we can pass in
the configuration elements that were previously hard coded. The primary
difference is it's possible to run a user specified subset of the test
cases.

Also removes references in the test definitions to specific user
directories.

There is a mostly duplicated version of the job runner files called
jenkins-drs-disabled-run.sh - as best I can determine these files are not
unified only because some of the nightly tests have not been confirmed to
work when the test executor is in a container.

(cherry picked from commit 3b269fa)
zjs pushed a commit that referenced this pull request Jul 27, 2018
@hickeng @zjs
Preserve quoting in test runner script (#8060) (#8151)
9c2c47c 
This is a follow on from #8047 and corrects the variable
quoting when passing though one script to another.

This has been tested via a branch build with multiple
test suites specified.

(cherry picked from commit 88d047f)
zjs pushed a commit that referenced this pull request Jul 27, 2018
@hickeng @zjs
Integration test script refinement and fixes (#8063) (#8151)
c7bca3e 
1. correct link construction for download of log bundles
2. correct custom test bed specified case in nimbus utils
3. clean generated log file name

The spec arg has been moved to the end to reduce the changes
necessary in calls to the Nimbus Deploy Testbed keyword that do
not specify a custom testbed file.

(cherry picked from commit 0f68789)
zjs pushed a commit that referenced this pull request Jul 27, 2018
@hickeng @zjs
Additional fixes to integration tests (#8069) (#8151)
5b5006e 
This builds on #8047 and #8063 to address some outstanding issues left by
that work. In the case of the hetrogeneous and NFS tests, it is test
updates necessitated by the prior Nimbus-Util.robot updates.

In the case of the VSAN tests it switches to using a testbed descriptor
based off the vic-simple-cluster spec instead of what appeared to be a
library spec, testng/testbeds/vsan.rb.

This disables the complex vSAN testsuite at this time as I could not infer
what the complex case was testing, and it is not in the test doc. It's
possible that it's testing multiple vSAN datastores and our ability to use
either but that's pure speculation.

(cherry picked from commit d533414)
zjs pushed a commit that referenced this pull request Jul 27, 2018
@hickeng @zjs
Updates to suite&dvs retry logic, increased predictability (#8074) (#…
b9a5480 
…8151)

This commit attempts to improve predictability by normalizing error
checking, retry approaches, pxe boot directory, and parameterizing
previously hard-coded build config. The latter is included because it
allows the caller to make reasonable estimates about how the build will
run rather than having to dig through test code.

Previously there were retries of different durations wrapped around the
outside of the Add Host To Distributed Switch keyword. If it may need
retrying then the operation should do so to avoid this cluster, and those
cases where a retry was not present.

Reduce sleep in inner retry loops to fail faster and adds a check for exit
code in addition to output parsing for faster fail. The inner loops should
be parameterized or removed completely in favour of the outer retry
logic unless there's significant improvement in success probability from
retrying at that level however this commit does not embark on that
work.

(cherry picked from commit 1387ab1)
zjs pushed a commit that referenced this pull request Jul 27, 2018
@hickeng @zjs
Update vsphere build numbers for scenario tests (#8136) (#8151)
e5303b5 
This updates the build numbers used for scenario test setup to the latest
update relases of the respective vSphere versions for 6.0, 6.5, and 6.7
Adds perma-links to the version/build mappings for future convenience.

(cherry picked from commit 3af7fa6)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

2 more reviewers

@DanielXiao DanielXiao DanielXiao approved these changes

@hickeng hickeng hickeng approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@zjs zjs

Labels

cla-not-required

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

9 participants

@zjs @DanielXiao @hickeng @vmwclabot @mhagen-vmware @jzt @rajanashok @andrewtchin @singhshwetaQE