Skip to content

[SDCICD-1729] Enable native osde2e S3 upload for test artifacts#519

Closed
YiqinZhang wants to merge 1 commit intoopenshift:mainfrom
YiqinZhang:sdcicd-1729
Closed

[SDCICD-1729] Enable native osde2e S3 upload for test artifacts#519
YiqinZhang wants to merge 1 commit intoopenshift:mainfrom
YiqinZhang:sdcicd-1729

Conversation

@YiqinZhang
Copy link
Copy Markdown
Contributor

@YiqinZhang YiqinZhang commented Jan 23, 2026

Configure e2e-template.yml to use native osde2e S3 upload for test artifacts instead of a sidecar container approach.

Removed

  • ❌ Sidecar container approach (aws-cli container)
  • ❌ Shared emptyDir volume between containers
  • ❌ Complex container coordination logic
  • ❌ Unnecessary configuration parameters (S3_UPLOAD_* variables)

Simplified

  • ✅ Single LOG_BUCKET parameter enables native S3 upload
  • ✅ Component name auto-derived from TEST_SUITES_YAML image
  • ✅ S3 path automatically organized: test-results/<component>/<date>/<job-id>/
  • ✅ Reuses existing AWS credentials (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)

Configuration

osde2e automatically handles S3 upload when LOG_BUCKET is set:

Environment Variable Value Source
LOG_BUCKET osde2e-loki-logs Template parameter
AWS_ACCESS_KEY_ID (from secret) Already configured
AWS_SECRET_ACCESS_KEY (from secret) Already configured
Component name osd-example-operator Auto-derived from TEST_SUITES_YAML
AWS Region us-east-1 Auto-default in osde2e

S3 Path Structure

Artifacts are automatically organized:

s3://osde2e-loki-logs/test-results/osd-example-operator/2026-01-27/<job-id>/
├── junit_<suffix>.xml
├── test_output.log
└── ... (other test artifacts)

SDCICD-1729

@ritmun
Copy link
Copy Markdown
Contributor

ritmun commented Jan 23, 2026

/hold for openshift/osde2e#3089

@openshift-ci openshift-ci Bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 23, 2026
@YiqinZhang
Copy link
Copy Markdown
Contributor Author

/unhold

@openshift-ci openshift-ci Bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 27, 2026
@openshift-ci
Copy link
Copy Markdown

openshift-ci Bot commented Jan 27, 2026

@YiqinZhang: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Comment thread test/e2e/e2e-template.yml
required: true
- name: LOG_BUCKET
value: 'osde2e-logs'
value: 'osde2e-loki-logs'
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the difference between these 2 buckets? are they in different accounts?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No big difference. osde2e-loki-logs is the existing bucket , using it just avoids creating a new one.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

osde2e-logs is an existing bucket as well, specified in all PD pipelines.

@YiqinZhang YiqinZhang requested a review from ritmun January 27, 2026 21:05
Comment thread test/e2e/e2e-template.yml
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest comparing this with boilerplate template and minimizing any non-necessary changes, as these will need to be updated in all PD pipelines

@YiqinZhang
Copy link
Copy Markdown
Contributor Author

#521

@YiqinZhang YiqinZhang closed this Jan 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants