Parameterize nightly script and remove user dir dependency by hickeng · Pull Request #8047 · vmware/vic

hickeng · 2018-06-13T19:40:46Z

Update the nightly logic to remove all dependence on specific user directories and content.
Update the Jenkins entry point scripts to allow for parameterization and customization of the run.

Of note is the almost complete duplication between the jenkins and upload scripts for drs_disabled and the nightlies. Reaching out to @anchal-agrawal this is because the nightlies were not yet migrated to use a container for running the test executor.

I'm not sure where we will actually go for this given a desire to have the executors be VCH containers themselves. We could have a single image for the job class, an image for the specific jobs, or a general executor image and very task specific images for the job steps. Planning required.

hickeng · 2018-06-13T19:46:58Z

tests/nightly/jenkins-nightly-run.sh

 ESX_67_VERSION="ob-8169922"
 VC_67_VERSION="ob-8217866"

+DEFAULT_TESTCASES="tests/manual-test-cases/Group5-Functional-Tests tests/manual-test-cases/Group13-vMotion tests/manual-test-cases/Group21-Registries tests/manual-test-cases/Group23-Future-Tests"


This was just moved from below - need to figure out if:
a. it's actually the full set of tests we should be running
b. why we don't have a nimbus-test-cases directory or similar for those expected to be run automatically

hickeng · 2018-06-13T19:53:27Z

tests/nightly/jenkins-nightly-run.sh

 fi
 target="$1"
-echo "Target cluster: "$target
+echo "Target cluster: ${target}"


This phrasing is terrible - this isn't a target cluster it's a vSphere version.

hickeng · 2018-06-13T20:03:53Z

tests/resources/Nimbus-Util.robot

+
    :FOR  ${IDX}  IN RANGE  1  5
-    \   ${out}=  Execute Command  ${NIMBUS_LOCATION} nimbus-testbeddeploy --lease 0.25 ${testbed}
+    \   ${out}=  Execute Command  ${NIMBUS_LOCATION} nimbus-testbeddeploy --lease 0.25 --testbedSpecRubyFile ./%{BUILD_TAG}/testbeds/${spec} ${args}


Only include the spec arguments if a spec was provided.

Instead I've included a default. This should be revisted based on

vic/tests/resources/Nimbus-Util.robot

Line 254 in e9345d6

${out}= Deploy Nimbus Testbed %{NIMBUS_USER} %{NIMBUS_PASSWORD} --plugin testng --lease 0.25 --noStatsDump --noSupportBundles --vcvaBuild ${VC_VERSION} --esxPxeDir ${ESX_VERSION} --esxBuild ${ESX_VERSION} --testbedName vcqa-vsan-simple-pxeBoot-vcva --runName ${name}

which doesn't provide a spec but does provide a testbedname - implies that a spec is not needed under some circumstances. It's unclear then why we are providing both a spec and a name in other locations - does this cause a merge of some kind?

hickeng · 2018-06-13T20:04:49Z

tests/resources/Nimbus-Util.robot

+    ${out}=  Execute Command  ${NIMBUS_LOCATION} rm -rf public_html/pxe/* %{BUILD_TAG}/testbeds
    Close connection

 Nimbus Cleanup


Add cleanup for BUILD_TAG directory

hickeng · 2018-06-13T20:05:33Z

tests/nightly/jenkins-nightly-run.sh

@@ -16,84 +16,75 @@
 ESX_60_VERSION="ob-5251623"


Add script usage doc.

This parameterizes the nightly job runner script so that we can pass in the configuration elements that were previously hard coded. The primary difference is it's possible to run a user specified subset of the test cases. Also removes references in the test definitions to specific user directories. There is a mostly duplicated version of the job runner files called jenkins-drs-disabled-run.sh - as best I can determine these files are not unified only because some of the nightly tests have not been confirmed to work when the test executor is in a container.

This is a follow on from #8047 and corrects the variable quoting when passing though one script to another. This has been tested via a branch build with multiple test suites specified.

This builds on vmware#8047 and vmware#8063 to address some outstanding issues left by that work. In the case of the hetrogeneous and NFS tests, it is test updates necessitated by the prior Nimbus-Util.robot updates. In the case of the VSAN tests it is addressing removal of a non-existent path and switching to use the related testbed spec.

This builds on #8047 and #8063 to address some outstanding issues left by that work. In the case of the hetrogeneous and NFS tests, it is test updates necessitated by the prior Nimbus-Util.robot updates. In the case of the VSAN tests it switches to using a testbed descriptor based off the vic-simple-cluster spec instead of what appeared to be a library spec, testng/testbeds/vsan.rb. This disables the complex vSAN testsuite at this time as I could not infer what the complex case was testing, and it is not in the test doc. It's possible that it's testing multiple vSAN datastores and our ability to use either but that's pure speculation.

This parameterizes the nightly job runner script so that we can pass in the configuration elements that were previously hard coded. The primary difference is it's possible to run a user specified subset of the test cases. Also removes references in the test definitions to specific user directories. There is a mostly duplicated version of the job runner files called jenkins-drs-disabled-run.sh - as best I can determine these files are not unified only because some of the nightly tests have not been confirmed to work when the test executor is in a container. (cherry picked from commit 3b269fa)

This is a follow on from vmware#8047 and corrects the variable quoting when passing though one script to another. This has been tested via a branch build with multiple test suites specified. (cherry picked from commit 88d047f)

This builds on vmware#8047 and vmware#8063 to address some outstanding issues left by that work. In the case of the hetrogeneous and NFS tests, it is test updates necessitated by the prior Nimbus-Util.robot updates. In the case of the VSAN tests it switches to using a testbed descriptor based off the vic-simple-cluster spec instead of what appeared to be a library spec, testng/testbeds/vsan.rb. This disables the complex vSAN testsuite at this time as I could not infer what the complex case was testing, and it is not in the test doc. It's possible that it's testing multiple vSAN datastores and our ability to use either but that's pure speculation. (cherry picked from commit d533414)

This parameterizes the nightly job runner script so that we can pass in the configuration elements that were previously hard coded. The primary difference is it's possible to run a user specified subset of the test cases. Also removes references in the test definitions to specific user directories. There is a mostly duplicated version of the job runner files called jenkins-drs-disabled-run.sh - as best I can determine these files are not unified only because some of the nightly tests have not been confirmed to work when the test executor is in a container. (cherry picked from commit 3b269fa)

This is a follow on from vmware#8047 and corrects the variable quoting when passing though one script to another. This has been tested via a branch build with multiple test suites specified. (cherry picked from commit 88d047f)

This builds on vmware#8047 and vmware#8063 to address some outstanding issues left by that work. In the case of the hetrogeneous and NFS tests, it is test updates necessitated by the prior Nimbus-Util.robot updates. In the case of the VSAN tests it switches to using a testbed descriptor based off the vic-simple-cluster spec instead of what appeared to be a library spec, testng/testbeds/vsan.rb. This disables the complex vSAN testsuite at this time as I could not infer what the complex case was testing, and it is not in the test doc. It's possible that it's testing multiple vSAN datastores and our ability to use either but that's pure speculation. (cherry picked from commit d533414)

This parameterizes the nightly job runner script so that we can pass in the configuration elements that were previously hard coded. The primary difference is it's possible to run a user specified subset of the test cases. Also removes references in the test definitions to specific user directories. There is a mostly duplicated version of the job runner files called jenkins-drs-disabled-run.sh - as best I can determine these files are not unified only because some of the nightly tests have not been confirmed to work when the test executor is in a container. (cherry picked from commit 3b269fa)

This is a follow on from #8047 and corrects the variable quoting when passing though one script to another. This has been tested via a branch build with multiple test suites specified. (cherry picked from commit 88d047f)

This builds on #8047 and #8063 to address some outstanding issues left by that work. In the case of the hetrogeneous and NFS tests, it is test updates necessitated by the prior Nimbus-Util.robot updates. In the case of the VSAN tests it switches to using a testbed descriptor based off the vic-simple-cluster spec instead of what appeared to be a library spec, testng/testbeds/vsan.rb. This disables the complex vSAN testsuite at this time as I could not infer what the complex case was testing, and it is not in the test doc. It's possible that it's testing multiple vSAN datastores and our ability to use either but that's pure speculation. (cherry picked from commit d533414)

hickeng requested a review from a team as a code owner June 13, 2018 19:40

vmwclabot added the cla-not-required label Jun 13, 2018

hickeng commented Jun 13, 2018

View reviewed changes

tests/nightly/jenkins-nightly-run.sh

@@ -16,84 +16,75 @@

ESX_60_VERSION="ob-5251623"

Copy link

Contributor Author

hickeng Jun 13, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add script usage doc.

hickeng changed the title ~~Post mikeh fix nightlies~~ Parameterize nightly script and remove user dir dependency Jun 13, 2018

hickeng force-pushed the post-mikeh-fix-nightlies branch 2 times, most recently from db59c4e to 19a6d5f Compare June 13, 2018 22:48

hickeng requested review from lcastellano and zjs June 14, 2018 00:19

zjs approved these changes Jun 14, 2018

View reviewed changes

hickeng force-pushed the post-mikeh-fix-nightlies branch from 19a6d5f to 5e836fb Compare June 14, 2018 17:28

zjs approved these changes Jun 14, 2018

View reviewed changes

hickeng force-pushed the post-mikeh-fix-nightlies branch 6 times, most recently from 62d546a to d246d2e Compare June 15, 2018 22:26

hickeng force-pushed the post-mikeh-fix-nightlies branch from d246d2e to 3a87280 Compare June 15, 2018 22:45

lcastellano approved these changes Jun 15, 2018

View reviewed changes

hickeng merged commit 3b269fa into vmware:master Jun 15, 2018

hickeng mentioned this pull request Jun 16, 2018

Preserve quoting in test runner script #8060

Merged

hickeng mentioned this pull request Jun 17, 2018

Additional fixes to integration tests #8069

Merged

zjs mentioned this pull request Jul 19, 2018

Cherry-pick: Changes to allow us to run nightly tests for 1.4.3 #8151

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Parameterize nightly script and remove user dir dependency#8047

Parameterize nightly script and remove user dir dependency#8047
hickeng merged 1 commit intovmware:masterfrom
hickeng:post-mikeh-fix-nightlies

hickeng commented Jun 13, 2018 •

edited

Loading

Uh oh!

hickeng Jun 13, 2018

Uh oh!

hickeng Jun 13, 2018

Uh oh!

hickeng Jun 13, 2018

Uh oh!

hickeng Jun 14, 2018

Uh oh!

hickeng Jun 13, 2018

Uh oh!

hickeng Jun 13, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

hickeng commented Jun 13, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hickeng Jun 13, 2018

Choose a reason for hiding this comment

Uh oh!

hickeng Jun 13, 2018

Choose a reason for hiding this comment

Uh oh!

hickeng Jun 13, 2018

Choose a reason for hiding this comment

Uh oh!

hickeng Jun 14, 2018

Choose a reason for hiding this comment

Uh oh!

hickeng Jun 13, 2018

Choose a reason for hiding this comment

Uh oh!

hickeng Jun 13, 2018

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

hickeng commented Jun 13, 2018 •

edited

Loading