Stepwise docker env by catsby · Pull Request #9292 · hashicorp/vault

catsby · 2020-06-23T12:02:10Z

This PR introduces the first environment for the Stepwise plugin testing framework, as well as 3 plugin tests that utilizes it:

logical/transit/stepwise_test.go
credential/userpass/stepwise_test.go
logical/aws/stepwise_test.go (requires valid TEST_AWS_ACCESS_KEY and TEST_AWS_SECRET_KEY environment variables)

With a local instance of Docker accessible, the tests can be ran using with the make testacc step:

$ make testacc TEST=./builtin/logical/transit TESTARGS="-count=1 -run=TestAcc"  
$ make testacc TEST=./builtin/credential/userpass TESTARGS="-count=1 -run=TestAcc"  
$ make testacc TEST=./builtin/logical/aws TESTARGS="-count=1 -run=TestAcc"  # requires TEST_AWS env vars

This is a work in progress which will continue beyond this PR, including tests for this environment which will follow later.

* master: Entity and alias counts (#9262) Token gauge metrics implementation. (#9239) mfa: fix import path on test file (#9303) doc: update vault helm enterprise image examples (#9299) raft: add support for using backend for ha_storage (#9193) Document new and previously undocumented telemetry metrics: (#9283) Improve the performance of snapshot installs by using rename (#9247) docs: add additional info around transform for tweak and template type (#9203) Update CHANGELOG.md CL++: Add go version to server message output

…during teardown

* master: changelog++ Update kerberos auth plugin (#9307) Ui/fix manage grid (#9309) plugins/openldap: update go mod (#9301) plugins/ssh: add diabled host key verification warning (#9304)

jasonodonnell

LGTM

pcman312 · 2020-06-25T20:58:11Z

+	return nil
+}
+
+func (rc *DockerCluster) setupCA(opts *DockerClusterOptions) error {


You don't need to do anything for this PR, but I'm curious if an existing library could be used here instead of doing all of the cert generation logic here? Maybe https://github.com/hashicorp/vault/blob/master/helper/testhelpers/certhelpers/cert_helpers.go ?

+1 on this.

pcman312 · 2020-06-25T21:02:06Z

+
+// Cleanup kills the container of the node
+func (n *DockerClusterNode) Cleanup() error {
+	return n.dockerAPI.ContainerKill(context.Background(), n.container.ID, "KILL")


Note for future iterations: this could potentially hang forever since there isn't any timeout or way of cancelling the underlying request since this is using context.Background()

Maybe we could add this via context.WithTimeout and use a relatively long value. Log something along the lines of "Waiting for cleanup to finish", and output the container ID if the cleanup times out so that users can clean them up manually.

Added a context.WithTimeout in b953396

haha thanks :D Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

Great catch, thanks Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>

…mber

catsby · 2020-06-26T22:52:18Z

test-go-race is failing on the main branch, I have the go-ahead to merge as-is

* add first stepwise test env, Docker, with example transit test * update transit stepwise test * add other tests that use stepwise * cleanup test, make names different than just 'transit' * return the stderr if compile fails with error * minor cleanups * minor cleanups * go mod vendor * cleanups * remove some extra code, and un-export some fields/methods * update vendor * remove reference to vault.CoreConfig, which really wasn't used anyway * update with go mod vendor * restore Precheck method to test cases * clean up some networking things; create networks with UUID, clean up during teardown * vendor stepwise * Update sdk/testing/stepwise/environments/docker/environment.go haha thanks :D Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com> * Update sdk/testing/stepwise/environments/docker/environment.go Great catch, thanks Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com> * fix redundant name * update error message in test * Update builtin/credential/userpass/stepwise_test.go More explicit error checking and responding Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com> * Update builtin/logical/aws/stepwise_test.go `test` -> `testFunc` Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com> * Update builtin/logical/transit/stepwise_test.go Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com> * fix typos * update error messages to provide clarity * Update sdk/testing/stepwise/environments/docker/environment.go Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com> * update error handling / collection in Teardown * panic if GenerateUUID returns an error * Update sdk/testing/stepwise/environments/docker/environment.go Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com> * Update builtin/credential/userpass/stepwise_test.go Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com> * Update builtin/logical/aws/stepwise_test.go Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com> * Update builtin/logical/transit/stepwise_test.go Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com> * Update sdk/testing/stepwise/environments/docker/environment.go Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com> * import ordering * standardize on dc from rc for cluster * lowercase name * CreateAPIClient -> NewAPIClient * testWait -> ensure * go mod cleanup * cleanups * move fields and method around * make start and dockerclusternode private; use better random serial number * use better random for SerialNumber * add a timeout to the context used for terminating the docker container * Use a constant for the Docker client version * rearrange import statements Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com> Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>

Co-authored-by: Dan Rivera <dan.rivera@hashicorp.com>

catsby added 7 commits June 23, 2020 06:50

add first stepwise test env, Docker, with example transit test

8f59d20

update transit stepwise test

743af07

add other tests that use stepwise

44792cd

cleanup test, make names different than just 'transit'

58c7c17

return the stderr if compile fails with error

85fa86e

minor cleanups

7a8e92b

minor cleanups

4489cac

catsby added this to the 1.5 milestone Jun 23, 2020

catsby marked this pull request as ready for review June 23, 2020 15:34

calvn self-requested a review June 23, 2020 18:21

catsby added 7 commits June 23, 2020 15:43

go mod vendor

0987f62

cleanups

3c5b844

remove some extra code, and un-export some fields/methods

32f8873

update vendor

17b3a2a

remove reference to vault.CoreConfig, which really wasn't used anyway

936c6b0

update with go mod vendor

719149c

jasonodonnell reviewed Jun 24, 2020

View reviewed changes

Comment thread builtin/credential/userpass/stepwise_test.go

jasonodonnell reviewed Jun 24, 2020

View reviewed changes

Comment thread builtin/logical/aws/stepwise_test.go

jasonodonnell reviewed Jun 24, 2020

View reviewed changes

Comment thread sdk/testing/stepwise/environments/docker/environment.go

jasonodonnell reviewed Jun 24, 2020

View reviewed changes

Comment thread sdk/testing/stepwise/environments/docker/environment.go Outdated

catsby added 4 commits June 24, 2020 16:19

restore Precheck method to test cases

de68f8a

clean up some networking things; create networks with UUID, clean up …

bb300a3

…during teardown

Merge branch 'master' into stepwise-docker-env

e38f6db

* master: changelog++ Update kerberos auth plugin (#9307) Ui/fix manage grid (#9309) plugins/openldap: update go mod (#9301) plugins/ssh: add diabled host key verification warning (#9304)

vendor stepwise

f487aa7

catsby requested a review from jasonodonnell June 25, 2020 12:11

jasonodonnell approved these changes Jun 25, 2020

View reviewed changes

pcman312 reviewed Jun 25, 2020

View reviewed changes

catsby and others added 2 commits June 25, 2020 17:00

Update sdk/testing/stepwise/environments/docker/environment.go

2cc9606

haha thanks :D Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

Update sdk/testing/stepwise/environments/docker/environment.go

b37910f

Great catch, thanks Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

catsby and others added 24 commits June 26, 2020 06:26

Update builtin/logical/transit/stepwise_test.go

c0cd87f

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

fix typos

17efb85

update error messages to provide clarity

3672dd9

Update sdk/testing/stepwise/environments/docker/environment.go

ae29f44

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

update error handling / collection in Teardown

f7bec5e

panic if GenerateUUID returns an error

1292a02

Update sdk/testing/stepwise/environments/docker/environment.go

5ee44cc

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

Update builtin/credential/userpass/stepwise_test.go

45c2219

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>

Update builtin/logical/aws/stepwise_test.go

e9eba5a

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>

Update builtin/logical/transit/stepwise_test.go

a7248e5

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>

Update sdk/testing/stepwise/environments/docker/environment.go

33a8840

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>

import ordering

6c7d2a6

standardize on dc from rc for cluster

4e548af

lowercase name

113085c

CreateAPIClient -> NewAPIClient

39bb77b

testWait -> ensure

61e1d39

go mod cleanup

0417e65

cleanups

efb823f

move fields and method around

813afbd

make start and dockerclusternode private; use better random serial nu…

863f2a1

…mber

use better random for SerialNumber

e382917

add a timeout to the context used for terminating the docker container

b953396

Use a constant for the Docker client version

e9b48a7

rearrange import statements

5467f25

pcman312 approved these changes Jun 26, 2020

View reviewed changes

calvn approved these changes Jun 26, 2020

View reviewed changes

catsby merged commit b3f0e9b into master Jun 26, 2020

pull Bot pushed a commit to Zezo-Ai/vault that referenced this pull request Sep 11, 2025

adding tests for plugin settings page (hashicorp#9292) (hashicorp#9296)

7a6d1b5

Co-authored-by: Dan Rivera <dan.rivera@hashicorp.com>

Conversation

catsby commented Jun 23, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jasonodonnell left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pcman312 Jun 25, 2020

Choose a reason for hiding this comment

Uh oh!

calvn Jun 25, 2020

Choose a reason for hiding this comment

Uh oh!

pcman312 Jun 25, 2020

Choose a reason for hiding this comment

Uh oh!

calvn Jun 25, 2020

Choose a reason for hiding this comment

Uh oh!

catsby Jun 26, 2020

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

catsby commented Jun 26, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

catsby commented Jun 23, 2020 •

edited

Loading