.github,test: Upload container logs during e2e runs as artifacts#2432
Conversation
Update the operatorsv1 API package import from v1 -> operatorsv1. Signed-off-by: timflannagan <timflannagan@gmail.com>
Update the $JUNIT_DIRECTORY environment varaible that's used throughout the e2e testing suite to the more generalized $ARTIFACTS_DIRECTORY which can be used as the base directory for various testing artifacts. Junit reports are now created implicitly whenever the $ARTIFACTS_DIRECTORY has been specified/non-empty. Signed-off-by: timflannagan <timflannagan@gmail.com>
…visioning Update the kind provisioner and collect container logs (and various other testing artifacts) before deprovisioning the kind cluster that tests were running. Signed-off-by: timflannagan <timflannagan@gmail.com>
| with: | ||
| go-version: '~1.16' | ||
| - run: make e2e-local E2E_NODES=2 JUNIT_DIRECTORY=./artifacts/ | ||
| - run: make e2e-local E2E_NODES=2 ARTIFACTS_DIR=./artifacts/ |
There was a problem hiding this comment.
Note: we'd need to also update the downstream o/release configuration for this environment variable as well: https://github.com/openshift/release/blob/master/ci-operator/config/openshift/operator-framework-olm/openshift-operator-framework-olm-release-4.10.yaml#L81.
| if junitDir := os.Getenv("JUNIT_DIRECTORY"); junitDir != "" { | ||
| junitReporter := reporters.NewJUnitReporter(path.Join(junitDir, fmt.Sprintf("junit_e2e_%02d.xml", config.GinkgoConfig.ParallelNode))) | ||
| // always configure a junit report when ARTIFACTS_DIR has been set | ||
| if artifactsDir := os.Getenv("ARTIFACTS_DIR"); artifactsDir != "" { |
There was a problem hiding this comment.
JUnit reports are now created implicitly whenever the $ARTIFACTS_DIRECTORY has been specified/non-empty.
|
Link to an action run that has the populated artifacts structure: https://github.com/operator-framework/operator-lifecycle-manager/actions/runs/1413705354 # download actions zip run
$ mkdir artifacts
$ unzip <...> -d artifacts
$ tree -d -L 2 artifacts
tree -d -L 2 artifacts/
artifacts/
├── junit
└── logs
├── kind-75drbpk9xgmfzpm5-control-plane
└── kind-9q22pms6dbzdjglt-control-plane |
timflannagan
force-pushed
the
e2e-collect-container-logs
branch
from
d6f3399 to
1d45f50
Compare
timflannagan
changed the title
tylerslaton
left a comment
tylerslaton
left a comment
There was a problem hiding this comment.
This looks really clean. I took a look at the artifact you linked and its great. We can talk about this off-pr but have we thought of how we may then use these artifacts in an automated way to detect regressions?
|
/lgtm |
|
@timflannagan thanks for the example run. |
|
/approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: kevinrizza, timflannagan The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
openshift-ci
Bot
added
the
approved
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
2 similar comments
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
Force (squash) merging this as CI is currently blocked and this can improve visibility while we sort through the existing list of failures. |
5 participants
Description of the change:
Upload the container logs from the various kind clusters that were provisioned during individual e2e runs as artifacts using the upload-artifact action.
Motivation for the change:
Currently, there's little visibility into why individual e2e runs failed. A quick, short-term win is to collect the individual container logs that were running on the provisioned kind cluster to the configured artifacts directory for GHA.
Reviewer Checklist
/doc