Support running tests in full VM's by adam-stokes · Pull Request #1740 · elastic/e2e-testing

adam-stokes · 2021-10-28T16:39:59Z

Orchestrates deploying pure vm's for both amd64/arm64, utilizing ansible. Windows support coming next in followup
Provides uniqueness with policies and integrations so that all test suites can run against a single deployed stack
Update the step definitions to be a little more platform agnostic, still more work to do there in followup pr's

Signed-off-by: Adam Stokes 51892+adam-stokes@users.noreply.github.com

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have run the Unit tests (make unit-test), and they are passing locally
I have run the End-2-End tests for the suite I'm working on, and they are passing locally
I have noticed new Go dependencies (run make notice in the proper directory)

How to test this PR locally

Please see .ci/README.md

Related issues

Closes https://github.com/elastic/observability-robots/issues/720

This gives us full vm's to run our e2e tests in. Supports both Linux amd64/arm64, and will support Windows. Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com>

mergify · 2021-10-28T16:40:26Z

This pull request does not have a backport label. Could you fix it @adam-stokes? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

v./d./d./d is the label to automatically backport to the 7./d branch. /d is the digit
NOTE: backport-skip has been added to this pull request.

kuisathaverat · 2021-10-28T16:46:03Z

-def checkTestSuite(Map parallelTasks = [:], Map item = [:]) {
-    def suite = item.suite
-    def platforms = item.platforms
+def sshexec(connection, cmd){


candidate to be a step in the library :D

kuisathaverat · 2021-10-28T16:46:12Z

+  sh "ssh -tt -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${env.WORKSPACE}/${env.BASE_DIR}/e2essh ${connection} -- '${cmd}'"
+}
+
+def scpr(connection, remote_src, local_dst){


candidate to be a step in the library :D

elasticmachine · 2021-10-28T17:42:27Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2021-11-30T16:05:37.301+0000
Duration: 43 min 18 sec
Commit: 6cd07fa

Test stats 🧪

Test	Results
Failed	0
Passed	86
Skipped	0
Total	86

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

adam-stokes · 2021-11-01T12:28:30Z

/test

adam-stokes · 2021-11-02T13:21:08Z

/test

mdelapenya · 2021-11-02T17:32:57Z

+/*
+ * Runs python in venv
+ */
+def pyrun(args){


Another candidate for the library?

Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com>

Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>

Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com>

The expression in the e2e-tests already handles both skip_tests and force_skip_git_checks. We make use of the skip_tests env here to skip those test suites if no files are changed Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com>

adam-stokes · 2021-11-30T15:06:02Z

/test

Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com>

…vm-tester-feat

* Support running e2e in VM's Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com> (cherry picked from commit 8a94011) # Conflicts: # .ci/Jenkinsfile

* Support running e2e in VM's Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com> (cherry picked from commit 8a94011) # Conflicts: # .ci/Jenkinsfile # .gitignore

* Support running e2e in VM's Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com> (cherry picked from commit 8a94011) # Conflicts: # .ci/.e2e-tests.yaml # .ci/Jenkinsfile # .gitignore # e2e/_suites/fleet/features/fleet_mode_agent.feature # e2e/_suites/fleet/features/preconfigured_policies.feature # e2e/_suites/fleet/fleet.go # go.sum

* Support running tests in full VM's (#1740) * Support running e2e in VM's Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com> (cherry picked from commit 8a94011) # Conflicts: # .ci/Jenkinsfile * Fix conflict Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> Co-authored-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com>

* Support running tests in full VM's (#1740) * Support running e2e in VM's Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com> (cherry picked from commit 8a94011) # Conflicts: # .ci/Jenkinsfile # .gitignore * Fix conflicts Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> Co-authored-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com>

* Support running tests in full VM's (#1740) * Support running e2e in VM's Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com> (cherry picked from commit 8a94011) # Conflicts: # .ci/.e2e-tests.yaml # .ci/Jenkinsfile # .gitignore # e2e/_suites/fleet/features/fleet_mode_agent.feature # e2e/_suites/fleet/features/preconfigured_policies.feature # e2e/_suites/fleet/fleet.go # go.sum * Fix conflicts Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> * fix: remove not implemented function call The backport added a reference to code implemented in: Test preconfigured agent policies and output (#1680) But this PR was not backported to 7.15 because there is no support to preconfigured policies. Co-authored-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>

otherwise it fails in the ephemeral workers relative path was added in elastic#1740

Support running e2e in VM's

d176f2d

This gives us full vm's to run our e2e tests in. Supports both Linux amd64/arm64, and will support Windows. Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com>

adam-stokes requested a review from mdelapenya October 28, 2021 16:39

mergify Bot added the backport-skip Skip notification from the automated backport with mergify label Oct 28, 2021

kuisathaverat reviewed Oct 28, 2021

View reviewed changes

Comment thread .ci/Jenkinsfile Outdated

kuisathaverat reviewed Oct 28, 2021

View reviewed changes

Comment thread .ci/Jenkinsfile Outdated

kuisathaverat reviewed Oct 28, 2021

View reviewed changes

Comment thread .ci/ansible/playbook.yml Outdated

This was referenced Nov 2, 2021

Step definitions need to be reworded to support multiple os/installtype matrix #1511

Closed

Enrolling on Windows is flaky #1557

Closed

Merge branch 'master' into vm-tester-feat

beae168

mdelapenya reviewed Nov 2, 2021

View reviewed changes

Comment thread .ci/.e2e-tests.yaml Outdated

Merge branch 'master' into vm-tester-feat

d141a7f

mdelapenya reviewed Nov 2, 2021

View reviewed changes

Comment thread .ci/Jenkinsfile Outdated

mdelapenya reviewed Nov 2, 2021

View reviewed changes

Comment thread .ci/Jenkinsfile Outdated

mdelapenya reviewed Nov 2, 2021

View reviewed changes

Comment thread .ci/Jenkinsfile Outdated

mdelapenya reviewed Nov 2, 2021

View reviewed changes

Comment thread .ci/Jenkinsfile Outdated

mdelapenya reviewed Nov 2, 2021

View reviewed changes

Comment thread .ci/ansible/roles/common/tasks/setup_test_script.yml Outdated

adam-stokes and others added 7 commits November 3, 2021 07:29

Merge branch 'master' into vm-tester-feat

eba6d47

bump log level to trace

320fe92

Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com>

Update .ci/Jenkinsfile

ef41720

Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>

Update .ci/Jenkinsfile

8d2e855

Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>

Update .ci/Jenkinsfile

2775cf7

Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>

use provision-stack as tag instead

91b96b8

Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com>

dont test ubi8

ab742ac

Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com>

adam-stokes added 2 commits November 30, 2021 07:50

move elasticcredentials into es endpoint struct

27cbd73

Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com>

make notice

9f7caeb

Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com>

This was referenced Nov 30, 2021

Support creating policy with optional integrations #1853

Closed

[automation] update elastic stack version for testing 8.0.0-5dc82a0c #1848

Merged

adam-stokes added 3 commits November 30, 2021 08:28

move force git check into e2e stage prior to checktestsuite

58cc2cf

Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com>

Merge branch 'master' into vm-tester-feat

cbfdf91

adam-stokes added 2 commits November 30, 2021 11:04

add force skip git checks on unit test stage

7994852

Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com>

Merge branch 'vm-tester-feat' of github.com:elastic/e2e-testing into …

6cd07fa

…vm-tester-feat

adam-stokes merged commit 8a94011 into master Nov 30, 2021

mergify Bot deleted the vm-tester-feat branch November 30, 2021 16:56

adam-stokes added backport-v7.15.0 Automated backport with mergify backport-v7.16.0 Automated backport with mergify backport-v8.0.0 Automated backport with mergify and removed backport-skip Skip notification from the automated backport with mergify labels Nov 30, 2021

mergify Bot mentioned this pull request Nov 30, 2021

[8.0](backport #1740) Support running tests in full VM's #1854

Merged

mergify Bot mentioned this pull request Nov 30, 2021

[7.16](backport #1740) Support running tests in full VM's #1855

Merged

mergify Bot mentioned this pull request Nov 30, 2021

[7.15](backport #1740) Support running tests in full VM's #1856

Merged

adam-stokes mentioned this pull request Nov 30, 2021

chore: run k8s and helm test suites on ARM #1471

Closed

8 tasks

v1v added a commit to v1v/e2e-testing that referenced this pull request Jun 16, 2022

e2e: use relative path

8c95942

otherwise it fails in the ephemeral workers relative path was added in elastic#1740

Conversation

adam-stokes commented Oct 28, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist

How to test this PR locally

Related issues

Uh oh!

mergify Bot commented Oct 28, 2021

Uh oh!

kuisathaverat Oct 28, 2021

Choose a reason for hiding this comment

Uh oh!

kuisathaverat Oct 28, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

elasticmachine commented Oct 28, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Build stats

Test stats 🧪

💚 Flaky test report

🤖 GitHub comments

Uh oh!

adam-stokes commented Nov 1, 2021

Uh oh!

adam-stokes commented Nov 2, 2021

Uh oh!

Uh oh!

Uh oh!

mdelapenya Nov 2, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

adam-stokes commented Nov 30, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

adam-stokes commented Oct 28, 2021 •

edited

Loading

elasticmachine commented Oct 28, 2021 •

edited

Loading