[SDCICD-1729] Add S3 upload capability for test artifacts#3089
[SDCICD-1729] Add S3 upload capability for test artifacts#3089YiqinZhang wants to merge 1 commit intoopenshift:mainfrom
Conversation
This PR adds native S3 upload functionality to osde2e for uploading test artifacts (JUnit XML, logs) after test execution. Features: - New s3upload package with configurable S3 upload - Auto-derives operator name from test image - Generates presigned URLs (24h expiry) for easy access - Organized S3 structure: <prefix>/<operator>/<date>/<job-id>/ Configuration: - S3_UPLOAD_ENABLED: Enable/disable S3 upload (default: true) - S3_UPLOAD_BUCKET: S3 bucket name (default: osde2e-loki-logs) - S3_UPLOAD_REGION: AWS region (default: us-east-1) - S3_UPLOAD_PREFIX: Path prefix (default: test-results) - S3_UPLOAD_OPERATOR_NAME: Override operator name (optional) - S3_UPLOAD_PRESIGNED_URL_EXPIRY: URL expiry hours (default: 24)
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: YiqinZhang The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
openshift-ci
Bot
added
the
approved
|
@YiqinZhang: all tests passed! Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
|
||
| // OperatorName is the name of the operator being tested (auto-derived from TEST_IMAGE if not set) | ||
| // Env: OPERATOR_NAME | ||
| OperatorName string |
There was a problem hiding this comment.
Test suite may not always be an operator. Can you call it "bucker_prefix" ? Does this need to be an env var?
| viper.SetDefault(Tests.EnableS3Upload, true) | ||
| _ = viper.BindEnv(Tests.EnableS3Upload, "ENABLE_S3_UPLOAD") | ||
|
|
||
| _ = viper.BindEnv(Tests.OperatorName, "OPERATOR_NAME") |
There was a problem hiding this comment.
Same question - if this is only used as bucket name prefix, does this need to be an env var?
| // 2. Derive from TEST_IMAGE or TestSuites | ||
| // 3. Fallback to "unknown-operator" | ||
| func deriveOperatorName() string { | ||
| // Priority 1: Explicit OPERATOR_NAME env var |
| @@ -0,0 +1,328 @@ | |||
| // Package s3upload provides functionality for uploading test artifacts to S3. | |||
| package s3upload | |||
There was a problem hiding this comment.
Why not sue existing s3 package under common/aws ?
2 participants
Add native S3 upload functionality to osde2e for uploading test artifacts (JUnit XML, logs) after test execution.
Features:
Configuration (env var):
S3_UPLOAD_ENABLEDtrueS3_UPLOAD_BUCKETosde2e-loki-logsS3_UPLOAD_REGIONus-east-1S3_UPLOAD_PREFIXtest-resultsS3_UPLOAD_OPERATOR_NAMES3_UPLOAD_PRESIGNED_URL_EXPIRY168(7 days)SDCICD-1729