Skip to content

24.8 S3 Azure Regression tests #571

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Enmk merged 5 commits into from
Jan 23, 2025
Merged

24.8 S3 Azure Regression tests #571

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
cac1ac4
Enable azure s3 regression tests
MyroTk Dec 18, 2024
7737ad8
Update regression hash
MyroTk Dec 18, 2024
63bb078
Update clickhouse-regression tests to 81564a41aee34004b96df755e777b06…
MyroTk Jan 14, 2025
82be061
Update regression hash and runner labels
MyroTk Jan 17, 2025
1303a26
Update release_branches.yml
MyroTk Jan 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .github/workflows/regression.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -506,7 +506,7 @@ jobs:
strategy:
fail-fast: false
matrix:
STORAGE: [minio, aws_s3, gcs]
STORAGE: [minio, aws_s3, gcs, azure]
needs: [runner_labels_setup]
runs-on: ${{ fromJson(needs.runner_labels_setup.outputs.runner_labels) }}
timeout-minutes: ${{ inputs.timeout_minutes }}
Expand Down Expand Up @@ -548,6 +548,9 @@ jobs:
--aws-s3-region ${{ secrets.REGRESSION_AWS_S3_REGION }}
--aws-s3-key-id ${{ secrets.REGRESSION_AWS_S3_KEY_ID }}
--aws-s3-access-key ${{ secrets.REGRESSION_AWS_S3_SECRET_ACCESS_KEY }}
--azure-account-name ${{ secrets.AZURE_ACCOUNT_NAME }}
--azure-storage-key ${{ secrets.AZURE_STORAGE_KEY }}
--azure-container ${{ secrets.AZURE_CONTAINER_NAME }}
--attr project="$GITHUB_REPOSITORY" project.id="$GITHUB_REPOSITORY_ID" package="${{ env.clickhouse_path }}" version="${{ env.version }}" user.name="$GITHUB_ACTOR" repository="https://github.com/Altinity/clickhouse-regression" commit.hash="$(git rev-parse HEAD)" job.id="$GITHUB_RUN_ID" job.url="$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID" arch="$(uname -i)"
${{ env.args }} || EXITCODE=$?;
.github/add_link_to_logs.sh;
Expand Down
74 changes: 37 additions & 37 deletions .github/workflows/release_branches.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ on: # yamllint disable-line rule:truthy

jobs:
RunConfig:
runs-on: [self-hosted, altinity-on-demand, altinity-type-cax11, altinity-image-arm-snapshot-22.04-arm, altinity-startup-snapshot, altinity-setup-none]
runs-on: [self-hosted, altinity-on-demand, altinity-style-checker-aarch64]
outputs:
data: ${{ steps.runconfig.outputs.CI_DATA }}
steps:
Expand Down Expand Up @@ -76,7 +76,7 @@ jobs:
secrets: inherit
with:
test_name: Compatibility check (release)
runner_type: altinity-on-demand, altinity-type-cpx51, altinity-image-x86-snapshot-22.04-amd, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester
data: ${{ needs.RunConfig.outputs.data }}
CompatibilityCheckAarch64:
needs: [RunConfig, BuilderDebAarch64]
Expand All @@ -85,7 +85,7 @@ jobs:
secrets: inherit
with:
test_name: Compatibility check (aarch64)
runner_type: altinity-on-demand, altinity-type-cax41, altinity-image-arm-snapshot-22.04-arm, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester-aarch64
data: ${{ needs.RunConfig.outputs.data }}
#########################################################################################
#################################### ORDINARY BUILDS ####################################
Expand Down Expand Up @@ -163,7 +163,7 @@ jobs:
secrets: inherit
with:
test_name: Docker server image
runner_type: altinity-on-demand, altinity-type-cpx51, altinity-image-x86-snapshot-22.04-amd, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester
data: ${{ needs.RunConfig.outputs.data }}
DockerKeeperImage:
needs: [RunConfig, BuilderDebRelease, BuilderDebAarch64]
Expand All @@ -172,7 +172,7 @@ jobs:
secrets: inherit
with:
test_name: Docker keeper image
runner_type: altinity-on-demand, altinity-type-cpx51, altinity-image-x86-snapshot-22.04-amd, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester
data: ${{ needs.RunConfig.outputs.data }}
############################################################################################
##################################### BUILD REPORTER #######################################
Expand All @@ -181,7 +181,7 @@ jobs:
# run report check for failed builds to indicate the CI error
if: ${{ !cancelled() && needs.RunConfig.result == 'success' && contains(fromJson(needs.RunConfig.outputs.data).jobs_data.jobs_to_do, 'Builds') }}
needs: [RunConfig, BuilderDebRelease, BuilderDebAarch64, BuilderDebAsan, BuilderDebUBsan, BuilderDebMsan, BuilderDebTsan, BuilderDebDebug]
runs-on: [self-hosted, altinity-on-demand, altinity-type-cax11, altinity-image-arm-snapshot-22.04-arm, altinity-startup-snapshot, altinity-setup-none]
runs-on: [self-hosted, altinity-on-demand, altinity-style-checker-aarch64]
steps:
- name: Check out repository code
uses: Altinity/checkout@19599efdf36c4f3f30eb55d5bb388896faea69f6
Expand All @@ -202,7 +202,7 @@ jobs:
needs:
- BuilderDebRelease
- BuilderDebAarch64
runs-on: [self-hosted, altinity-on-demand, altinity-type-cax11, altinity-image-arm-snapshot-22.04-arm, altinity-startup-snapshot, altinity-setup-none]
runs-on: [self-hosted, altinity-on-demand, altinity-style-checker-aarch64]
steps:
- name: Debug
run: |
Expand Down Expand Up @@ -236,7 +236,7 @@ jobs:
secrets: inherit
with:
test_name: Install packages (release)
runner_type: altinity-on-demand, altinity-type-cpx51, altinity-image-x86-snapshot-22.04-amd, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester
data: ${{ needs.RunConfig.outputs.data }}
run_command: |
python3 install_check.py "$CHECK_NAME"
Expand All @@ -247,7 +247,7 @@ jobs:
secrets: inherit
with:
test_name: Install packages (aarch64)
runner_type: altinity-on-demand, altinity-type-cax41, altinity-image-arm-snapshot-22.04-arm, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester-aarch64
data: ${{ needs.RunConfig.outputs.data }}
run_command: |
python3 install_check.py "$CHECK_NAME"
Expand All @@ -261,7 +261,7 @@ jobs:
secrets: inherit
with:
test_name: Stateless tests (release)
runner_type: altinity-on-demand, altinity-type-cpx51, altinity-in-ash,altinity-image-x86-snapshot-22.04-amd, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester
data: ${{ needs.RunConfig.outputs.data }}
FunctionalStatelessTestAarch64:
needs: [RunConfig, BuilderDebAarch64]
Expand All @@ -270,7 +270,7 @@ jobs:
secrets: inherit
with:
test_name: Stateless tests (aarch64)
runner_type: altinity-on-demand, altinity-type-cax41, altinity-image-arm-snapshot-22.04-arm, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester-aarch64
data: ${{ needs.RunConfig.outputs.data }}
FunctionalStatelessTestAsan:
needs: [RunConfig, BuilderDebAsan]
Expand All @@ -279,7 +279,7 @@ jobs:
secrets: inherit
with:
test_name: Stateless tests (asan)
runner_type: altinity-on-demand, altinity-type-cpx51, altinity-image-x86-snapshot-22.04-amd, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester
data: ${{ needs.RunConfig.outputs.data }}
FunctionalStatelessTestTsan:
needs: [RunConfig, BuilderDebTsan]
Expand All @@ -288,7 +288,7 @@ jobs:
secrets: inherit
with:
test_name: Stateless tests (tsan)
runner_type: altinity-on-demand, altinity-type-cpx51, altinity-image-x86-snapshot-22.04-amd, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester
data: ${{ needs.RunConfig.outputs.data }}
FunctionalStatelessTestMsan:
needs: [RunConfig, BuilderDebMsan]
Expand All @@ -297,7 +297,7 @@ jobs:
secrets: inherit
with:
test_name: Stateless tests (msan)
runner_type: altinity-on-demand, altinity-type-cpx51, altinity-image-x86-snapshot-22.04-amd, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester
data: ${{ needs.RunConfig.outputs.data }}
FunctionalStatelessTestUBsan:
needs: [RunConfig, BuilderDebUBsan]
Expand All @@ -306,7 +306,7 @@ jobs:
secrets: inherit
with:
test_name: Stateless tests (ubsan)
runner_type: altinity-on-demand, altinity-type-cpx51, altinity-image-x86-snapshot-22.04-amd, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester
data: ${{ needs.RunConfig.outputs.data }}
FunctionalStatelessTestDebug:
needs: [RunConfig, BuilderDebDebug]
Expand All @@ -315,7 +315,7 @@ jobs:
secrets: inherit
with:
test_name: Stateless tests (debug)
runner_type: altinity-on-demand, altinity-type-cpx51, altinity-image-x86-snapshot-22.04-amd, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester
data: ${{ needs.RunConfig.outputs.data }}
##############################################################################################
############################ FUNCTIONAl STATEFUL TESTS #######################################
Expand All @@ -327,7 +327,7 @@ jobs:
secrets: inherit
with:
test_name: Stateful tests (release)
runner_type: altinity-on-demand, altinity-type-cpx51, altinity-image-x86-snapshot-22.04-amd, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester
data: ${{ needs.RunConfig.outputs.data }}
FunctionalStatefulTestAarch64:
needs: [RunConfig, BuilderDebAarch64]
Expand All @@ -336,7 +336,7 @@ jobs:
secrets: inherit
with:
test_name: Stateful tests (aarch64)
runner_type: altinity-on-demand, altinity-type-cax41, altinity-image-arm-snapshot-22.04-arm, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester-aarch64
data: ${{ needs.RunConfig.outputs.data }}
FunctionalStatefulTestAsan:
needs: [RunConfig, BuilderDebAsan]
Expand All @@ -345,7 +345,7 @@ jobs:
secrets: inherit
with:
test_name: Stateful tests (asan)
runner_type: altinity-on-demand, altinity-type-cpx51, altinity-image-x86-snapshot-22.04-amd, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester
data: ${{ needs.RunConfig.outputs.data }}
FunctionalStatefulTestTsan:
needs: [RunConfig, BuilderDebTsan]
Expand All @@ -354,7 +354,7 @@ jobs:
secrets: inherit
with:
test_name: Stateful tests (tsan)
runner_type: altinity-on-demand, altinity-type-cpx51, altinity-image-x86-snapshot-22.04-amd, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester
data: ${{ needs.RunConfig.outputs.data }}
FunctionalStatefulTestMsan:
needs: [RunConfig, BuilderDebMsan]
Expand All @@ -363,7 +363,7 @@ jobs:
secrets: inherit
with:
test_name: Stateful tests (msan)
runner_type: altinity-on-demand, altinity-type-cpx51, altinity-image-x86-snapshot-22.04-amd, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester
data: ${{ needs.RunConfig.outputs.data }}
FunctionalStatefulTestUBsan:
needs: [RunConfig, BuilderDebUBsan]
Expand All @@ -372,7 +372,7 @@ jobs:
secrets: inherit
with:
test_name: Stateful tests (ubsan)
runner_type: altinity-on-demand, altinity-type-cpx51, altinity-image-x86-snapshot-22.04-amd, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester
data: ${{ needs.RunConfig.outputs.data }}
FunctionalStatefulTestDebug:
needs: [RunConfig, BuilderDebDebug]
Expand All @@ -381,7 +381,7 @@ jobs:
secrets: inherit
with:
test_name: Stateful tests (debug)
runner_type: altinity-on-demand, altinity-type-cpx51, altinity-image-x86-snapshot-22.04-amd, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester
data: ${{ needs.RunConfig.outputs.data }}
##############################################################################################
######################################### STRESS TESTS #######################################
Expand All @@ -393,7 +393,7 @@ jobs:
secrets: inherit
with:
test_name: Stress test (asan)
runner_type: altinity-on-demand, altinity-type-cpx51, altinity-image-x86-snapshot-22.04-amd, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester
data: ${{ needs.RunConfig.outputs.data }}
StressTestTsan:
needs: [RunConfig, BuilderDebTsan]
Expand All @@ -402,7 +402,7 @@ jobs:
secrets: inherit
with:
test_name: Stress test (tsan)
runner_type: altinity-on-demand, altinity-type-cpx51, altinity-image-x86-snapshot-22.04-amd, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester
data: ${{ needs.RunConfig.outputs.data }}
StressTestMsan:
needs: [RunConfig, BuilderDebMsan]
Expand All @@ -411,7 +411,7 @@ jobs:
secrets: inherit
with:
test_name: Stress test (msan)
runner_type: altinity-on-demand, altinity-type-cpx51, altinity-image-x86-snapshot-22.04-amd, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester
data: ${{ needs.RunConfig.outputs.data }}
StressTestUBsan:
needs: [RunConfig, BuilderDebUBsan]
Expand All @@ -420,7 +420,7 @@ jobs:
secrets: inherit
with:
test_name: Stress test (ubsan)
runner_type: altinity-on-demand, altinity-type-cpx51, altinity-image-x86-snapshot-22.04-amd, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester
data: ${{ needs.RunConfig.outputs.data }}
StressTestDebug:
needs: [RunConfig, BuilderDebDebug]
Expand All @@ -429,7 +429,7 @@ jobs:
secrets: inherit
with:
test_name: Stress test (debug)
runner_type: altinity-on-demand, altinity-type-cpx51, altinity-image-x86-snapshot-22.04-amd, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester
data: ${{ needs.RunConfig.outputs.data }}
#############################################################################################
############################# INTEGRATION TESTS #############################################
Expand All @@ -441,7 +441,7 @@ jobs:
secrets: inherit
with:
test_name: Integration tests (asan)
runner_type: altinity-on-demand, altinity-type-cpx51, altinity-image-x86-snapshot-22.04-amd, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester
data: ${{ needs.RunConfig.outputs.data }}
IntegrationTestsAnalyzerAsan:
needs: [RunConfig, BuilderDebAsan]
Expand All @@ -450,7 +450,7 @@ jobs:
secrets: inherit
with:
test_name: Integration tests (asan, old analyzer)
runner_type: altinity-on-demand, altinity-type-cpx51, altinity-image-x86-snapshot-22.04-amd, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester
data: ${{ needs.RunConfig.outputs.data }}
IntegrationTestsTsan:
needs: [RunConfig, BuilderDebTsan]
Expand All @@ -459,7 +459,7 @@ jobs:
secrets: inherit
with:
test_name: Integration tests (tsan)
runner_type: altinity-on-demand, altinity-type-cpx51, altinity-image-x86-snapshot-22.04-amd, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester
data: ${{ needs.RunConfig.outputs.data }}
IntegrationTestsRelease:
needs: [RunConfig, BuilderDebRelease]
Expand All @@ -468,7 +468,7 @@ jobs:
secrets: inherit
with:
test_name: Integration tests (release)
runner_type: altinity-on-demand, altinity-type-cpx51, altinity-image-x86-snapshot-22.04-amd, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester
data: ${{ needs.RunConfig.outputs.data }}
#############################################################################################
##################################### REGRESSION TESTS ######################################
Expand All @@ -480,7 +480,7 @@ jobs:
secrets: inherit
with:
runner_type: altinity-on-demand, altinity-type-cpx51, altinity-image-x86-app-docker-ce, altinity-setup-regression
commit: 634042960dd157489199ca0f9d028c7fd1e6adb0
commit: 18c56fe46fae827748c557e9ebda0599c11b0a55
arch: release
build_sha: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
timeout_minutes: 300
Expand All @@ -491,7 +491,7 @@ jobs:
secrets: inherit
with:
runner_type: altinity-on-demand, altinity-type-cax41, altinity-image-arm-app-docker-ce, altinity-setup-regression
commit: 634042960dd157489199ca0f9d028c7fd1e6adb0
commit: 18c56fe46fae827748c557e9ebda0599c11b0a55
arch: aarch64
build_sha: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
timeout_minutes: 300
Expand All @@ -502,7 +502,7 @@ jobs:
secrets: inherit
with:
test_name: Sign release
runner_type: altinity-on-demand, altinity-type-cpx51, altinity-image-x86-snapshot-22.04-amd, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester
data: ${{ needs.RunConfig.outputs.data }}
SignAarch64:
needs: [RunConfig, BuilderDebAarch64]
Expand All @@ -511,7 +511,7 @@ jobs:
secrets: inherit
with:
test_name: Sign aarch64
runner_type: altinity-on-demand, altinity-type-cpx51, altinity-image-x86-snapshot-22.04-amd, altinity-startup-snapshot, altinity-setup-none
runner_type: altinity-on-demand, altinity-func-tester
data: ${{ needs.RunConfig.outputs.data }}
FinishCheck:
if: ${{ !cancelled() }}
Expand Down Expand Up @@ -548,7 +548,7 @@ jobs:
- RegressionTestsRelease
- RegressionTestsAarch64
- SignRelease
runs-on: [self-hosted, altinity-on-demand, altinity-type-cax11, altinity-image-arm-snapshot-22.04-arm, altinity-startup-snapshot, altinity-setup-none]
runs-on: [self-hosted, altinity-on-demand, altinity-style-checker-aarch64]
steps:
- name: Check out repository code
uses: Altinity/checkout@19599efdf36c4f3f30eb55d5bb388896faea69f6
Expand Down
Loading