Test preconfigured agent policies and output by nchaulet · Pull Request #1680 · elastic/e2e-testing

nchaulet · 2021-10-19T00:10:14Z

Description

Related to elastic/kibana#114285

I changed the preconfigured policies test this test:

define a preconfigured policy and a preconfigured output
enroll an agent to that preconfigured policy
verify the agent is running the policy

I made some change to the test runner:

I added a step to select an enrollment token
I changed the behavior of the kibana profile step to have the kibana profile use during the whole scenario.

elasticmachine · 2021-10-19T00:48:17Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2021-10-21T11:34:33.802+0000
Duration: 42 min 39 sec
Commit: 938da65

Test stats 🧪

Test	Results
Failed	0
Passed	242
Skipped	0
Total	242

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

mdelapenya · 2021-10-19T08:11:30Z

This is a great PR @nchaulet!! Thanks for taking the time in contributing the tests 👏 As soon you mark it as ready for review, I'll accept it, as it LGTM. Please see my previous comments, but they are non-functional related.

nchaulet · 2021-10-19T19:45:38Z

/test

mdelapenya · 2021-10-20T08:20:20Z

-Examples: Centos
-| os     |
-| centos |
+  And the agent run the "Test preconfigured policy" policy


Wdyt about renaming the step to:

And the agent uses the "Test preconfigured policy" policy

Using run seems very low-level, and we must be close to the business when defining steps. Unless running a policy is part of the vocabulary, of course

Oh I see, we are using the agent uses "A" policy for setting the policy, and the agent runs the "A" policy for checking it indeed was produced, right?

If this is business-correct, then I'm OK too 😃

I can probably change the use step by the agent uses enrollment token from "A" and run step by something like the agent is enrolled into "A" policy it's probably less confusion

nchaulet · 2021-10-20T13:11:03Z

I do not think we should target 7.15 version as the preconfigured output only appear in 7.16

mdelapenya · 2021-10-20T14:30:18Z

  When the "elastic-agent" process is in the "started" state on the host
  Then the agent is listed in Fleet as "online"
-  And the agent run the "Test preconfigured policy" policy
+  And the agent is enrolled into "Test preconfigured policy" policy


I love this step name! 😄

mdelapenya

I think we can merge it once the end-of-file issue in the feature file is fixed (pre-commit run end-of-file-fixer --files ./...).

The other 4 errors appeared today and seems related to heartbeat

Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>

nchaulet · 2021-10-20T16:05:04Z

/test

mdelapenya · 2021-10-20T17:35:36Z

Closes #1680

(cherry picked from commit 56ce765)

(cherry picked from commit 56ce765) Co-authored-by: Nicolas Chaulet <nicolas.chaulet@elastic.co> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>

The backport added a reference to code implemented in: Test preconfigured agent policies and output (#1680) But this PR was not backported to 7.15 because there is no support to preconfigured policies.

* Support running tests in full VM's (#1740) * Support running e2e in VM's Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com> (cherry picked from commit 8a94011) # Conflicts: # .ci/.e2e-tests.yaml # .ci/Jenkinsfile # .gitignore # e2e/_suites/fleet/features/fleet_mode_agent.feature # e2e/_suites/fleet/features/preconfigured_policies.feature # e2e/_suites/fleet/fleet.go # go.sum * Fix conflicts Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> * fix: remove not implemented function call The backport added a reference to code implemented in: Test preconfigured agent policies and output (#1680) But this PR was not backported to 7.15 because there is no support to preconfigured policies. Co-authored-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>

Test preconfiguration

30caca2

nchaulet self-assigned this Oct 19, 2021

mdelapenya reviewed Oct 19, 2021

View reviewed changes

Comment thread e2e/_suites/fleet/fleet.go

mdelapenya reviewed Oct 19, 2021

View reviewed changes

Comment thread e2e/_suites/fleet/fleet.go

nchaulet added 3 commits October 19, 2021 10:27

Add to CI

acbcbf1

add to CI take 2

b4c5fa1

Fix tests

ecc07cc

nchaulet marked this pull request as ready for review October 19, 2021 17:05

mdelapenya added v7.15.0 v7.16.0 labels Oct 20, 2021

mdelapenya reviewed Oct 20, 2021

View reviewed changes

Comment thread e2e/_suites/fleet/features/preconfigured_policies.feature

mdelapenya reviewed Oct 20, 2021

View reviewed changes

Comment thread e2e/_suites/fleet/fleet.go Outdated

Fix after code review

d343cba

nchaulet requested a review from mdelapenya October 20, 2021 13:10

nchaulet removed the v7.15.0 label Oct 20, 2021

mdelapenya reviewed Oct 20, 2021

View reviewed changes

mdelapenya approved these changes Oct 20, 2021

View reviewed changes

mdelapenya reviewed Oct 20, 2021

View reviewed changes

Comment thread e2e/_suites/fleet/features/preconfigured_policies.feature Outdated

nchaulet and others added 2 commits October 20, 2021 10:49

Update e2e/_suites/fleet/features/preconfigured_policies.feature

a3fe56c

Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>

Merge branch 'master' into feature-test-preconfigure

36bd661

nchaulet and others added 3 commits October 20, 2021 14:04

comment centos

93371c9

Merge branch 'master' into feature-test-preconfigure

b16d6b8

Merge branch 'master' into feature-test-preconfigure

938da65

nchaulet merged commit 56ce765 into master Oct 21, 2021

nchaulet deleted the feature-test-preconfigure branch October 21, 2021 12:29

mergify Bot pushed a commit that referenced this pull request Oct 21, 2021

Test preconfigured agent policies and output (#1680)

76b2f4a

(cherry picked from commit 56ce765)

mergify Bot mentioned this pull request Oct 21, 2021

[7.x](backport #1680) Test preconfigured agent policies and output #1701

Merged

nchaulet mentioned this pull request Oct 21, 2021

Add tests with preconfigured policies #1107

Closed

mdelapenya mentioned this pull request Nov 4, 2021

fix: remove preconfigured policies from 7.15 #1774

Merged

8 tasks

nchaulet mentioned this pull request Dec 8, 2021

[Fleet] Test ticket for: Allow to preconfigure (same cluster) alternative ES outputs elastic/kibana#114285

Closed

Conversation

nchaulet commented Oct 19, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Uh oh!

elasticmachine commented Oct 19, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Build stats

Test stats 🧪

💚 Flaky test report

🤖 GitHub comments

Uh oh!

Uh oh!

Uh oh!

mdelapenya commented Oct 19, 2021

Uh oh!

nchaulet commented Oct 19, 2021

Uh oh!

Uh oh!

mdelapenya Oct 20, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mdelapenya Oct 20, 2021

Choose a reason for hiding this comment

Uh oh!

nchaulet Oct 20, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nchaulet commented Oct 20, 2021

Uh oh!

mdelapenya Oct 20, 2021

Choose a reason for hiding this comment

Uh oh!

mdelapenya left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nchaulet commented Oct 20, 2021

Uh oh!

mdelapenya commented Oct 20, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

nchaulet commented Oct 19, 2021 •

edited

Loading

elasticmachine commented Oct 19, 2021 •

edited

Loading

mdelapenya Oct 20, 2021 •

edited

Loading

mdelapenya left a comment •

edited

Loading