Skip to content

CI: Consolidate test matrix configurations into ci/test-matrix.json with hard-coded values, optimized checkout, and prepared Windows self-hosted runner migration #889

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
leofang merged 13 commits into from
Sep 5, 2025
Merged

CI: Consolidate test matrix configurations into ci/test-matrix.json with hard-coded values, optimized checkout, and prepared Windows self-hosted runner migration #889

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
2392641
Initial plan
Copilot Aug 22, 2025
70833d1
Consolidate test matrices from workflows into ci/test-matrix.json
Copilot Aug 22, 2025
27232d6
Hard-code all GPU and ARCH values in test-matrix.json with 6 fields p…
Copilot Aug 22, 2025
111875a
Update Windows test matrix with a100 GPU and latest-1 driver, configu…
Copilot Aug 24, 2025
eed0b71
fix
leofang Aug 24, 2025
69875a3
Revert eed0b71 and change Windows DRIVER from latest-1 to latest
Copilot Aug 24, 2025
ae93e2b
Add proxy cache setup to Windows workflow for self-hosted runners
Copilot Aug 24, 2025
64141b5
Remove Git for Windows and gh CLI installation steps, add T4 GPU supp…
Copilot Aug 25, 2025
c5df9d0
Set fetch-depth: 1 for checkout steps and favor L4/T4 over A100 GPUs …
Copilot Aug 25, 2025
8afb850
Revert Windows workflow to GitHub-hosted runners with TODO comments f…
Copilot Sep 5, 2025
a9b97fc
[pre-commit.ci] auto code formatting
pre-commit-ci[bot] Sep 5, 2025
905ad53
Merge branch 'main' into copilot/fix-888
leofang Sep 5, 2025
1ad9d18
Revert Win runner name change for now
leofang Sep 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 21 additions & 64 deletions .github/workflows/test-wheel-linux.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,86 +34,43 @@ jobs:
outputs:
MATRIX: ${{ steps.compute-matrix.outputs.MATRIX }}
steps:
- name: Checkout ${{ github.event.repository.name }}
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
fetch-depth: 1

- name: Validate Test Type
run: |
if [[ "$BUILD_TYPE" != "pull-request" ]] && [[ "$BUILD_TYPE" != "nightly" ]] && [[ "$BUILD_TYPE" != "branch" ]]; then
echo "Invalid build type! Must be one of 'nightly', 'pull-request', or 'branch'."
exit 1
fi

- name: Compute Python Test Matrix
id: compute-matrix
run: |
# Set a default GPU based upon architecture.
gpu="l4"
if [[ "${ARCH}" == "arm64" ]]; then
gpu="a100"
fi
# Add a special entry for the H100 runner on amd64.
special_runner=""
if [[ "${ARCH}" == "amd64" ]]; then
special_runner="- { ARCH: ${ARCH}, PY_VER: '3.13', CUDA_VER: '13.0.0', LOCAL_CTK: '1', GPU: 'H100', DRIVER: 'latest' }"
fi

# Please keep the matrices sorted in ascending order by the following:
#
# [PY_VER, CUDA_VER, LOCAL_CTK, GPU, DRIVER]
#
# Note that DRIVER: `earliest` does not work with CUDA 12.9.0 and LOCAL_CTK: 0 does not work with CUDA 12.0.1.
#
export MATRICES="
pull-request:
- { ARCH: ${ARCH}, PY_VER: '3.9', CUDA_VER: '12.9.0', LOCAL_CTK: '0', GPU: ${gpu}, DRIVER: 'latest' }
- { ARCH: ${ARCH}, PY_VER: '3.9', CUDA_VER: '13.0.0', LOCAL_CTK: '1', GPU: ${gpu}, DRIVER: 'latest' }
- { ARCH: ${ARCH}, PY_VER: '3.10', CUDA_VER: '12.9.0', LOCAL_CTK: '1', GPU: ${gpu}, DRIVER: 'latest' }
- { ARCH: ${ARCH}, PY_VER: '3.10', CUDA_VER: '13.0.0', LOCAL_CTK: '0', GPU: ${gpu}, DRIVER: 'latest' }
- { ARCH: ${ARCH}, PY_VER: '3.11', CUDA_VER: '12.9.0', LOCAL_CTK: '0', GPU: ${gpu}, DRIVER: 'latest' }
- { ARCH: ${ARCH}, PY_VER: '3.11', CUDA_VER: '13.0.0', LOCAL_CTK: '1', GPU: ${gpu}, DRIVER: 'latest' }
- { ARCH: ${ARCH}, PY_VER: '3.12', CUDA_VER: '12.9.0', LOCAL_CTK: '1', GPU: ${gpu}, DRIVER: 'latest' }
- { ARCH: ${ARCH}, PY_VER: '3.12', CUDA_VER: '13.0.0', LOCAL_CTK: '0', GPU: ${gpu}, DRIVER: 'latest' }
- { ARCH: ${ARCH}, PY_VER: '3.13', CUDA_VER: '12.9.0', LOCAL_CTK: '0', GPU: ${gpu}, DRIVER: 'latest' }
- { ARCH: ${ARCH}, PY_VER: '3.13', CUDA_VER: '13.0.0', LOCAL_CTK: '1', GPU: ${gpu}, DRIVER: 'latest' }
${special_runner}
nightly:
- { ARCH: ${ARCH}, PY_VER: '3.9', CUDA_VER: '11.8.0', LOCAL_CTK: '0', GPU: ${gpu}, DRIVER: 'earliest' }
- { ARCH: ${ARCH}, PY_VER: '3.9', CUDA_VER: '11.8.0', LOCAL_CTK: '1', GPU: ${gpu}, DRIVER: 'latest' }
- { ARCH: ${ARCH}, PY_VER: '3.9', CUDA_VER: '12.0.1', LOCAL_CTK: '1', GPU: ${gpu}, DRIVER: 'latest' }
- { ARCH: ${ARCH}, PY_VER: '3.9', CUDA_VER: '12.9.0', LOCAL_CTK: '0', GPU: ${gpu}, DRIVER: 'latest' }
- { ARCH: ${ARCH}, PY_VER: '3.9', CUDA_VER: '12.9.0', LOCAL_CTK: '1', GPU: ${gpu}, DRIVER: 'latest' }
- { ARCH: ${ARCH}, PY_VER: '3.10', CUDA_VER: '11.8.0', LOCAL_CTK: '0', GPU: ${gpu}, DRIVER: 'earliest' }
- { ARCH: ${ARCH}, PY_VER: '3.10', CUDA_VER: '11.8.0', LOCAL_CTK: '1', GPU: ${gpu}, DRIVER: 'latest' }
- { ARCH: ${ARCH}, PY_VER: '3.10', CUDA_VER: '12.0.1', LOCAL_CTK: '1', GPU: ${gpu}, DRIVER: 'latest' }
- { ARCH: ${ARCH}, PY_VER: '3.10', CUDA_VER: '12.9.0', LOCAL_CTK: '0', GPU: ${gpu}, DRIVER: 'latest' }
- { ARCH: ${ARCH}, PY_VER: '3.10', CUDA_VER: '12.9.0', LOCAL_CTK: '1', GPU: ${gpu}, DRIVER: 'latest' }
- { ARCH: ${ARCH}, PY_VER: '3.11', CUDA_VER: '11.8.0', LOCAL_CTK: '0', GPU: ${gpu}, DRIVER: 'earliest' }
- { ARCH: ${ARCH}, PY_VER: '3.11', CUDA_VER: '11.8.0', LOCAL_CTK: '1', GPU: ${gpu}, DRIVER: 'latest' }
- { ARCH: ${ARCH}, PY_VER: '3.11', CUDA_VER: '12.0.1', LOCAL_CTK: '1', GPU: ${gpu}, DRIVER: 'latest' }
- { ARCH: ${ARCH}, PY_VER: '3.11', CUDA_VER: '12.9.0', LOCAL_CTK: '0', GPU: ${gpu}, DRIVER: 'latest' }
- { ARCH: ${ARCH}, PY_VER: '3.11', CUDA_VER: '12.9.0', LOCAL_CTK: '1', GPU: ${gpu}, DRIVER: 'latest' }
- { ARCH: ${ARCH}, PY_VER: '3.12', CUDA_VER: '11.8.0', LOCAL_CTK: '0', GPU: ${gpu}, DRIVER: 'earliest' }
- { ARCH: ${ARCH}, PY_VER: '3.12', CUDA_VER: '11.8.0', LOCAL_CTK: '1', GPU: ${gpu}, DRIVER: 'latest' }
- { ARCH: ${ARCH}, PY_VER: '3.12', CUDA_VER: '12.0.1', LOCAL_CTK: '1', GPU: ${gpu}, DRIVER: 'latest' }
- { ARCH: ${ARCH}, PY_VER: '3.12', CUDA_VER: '12.9.0', LOCAL_CTK: '0', GPU: ${gpu}, DRIVER: 'latest' }
- { ARCH: ${ARCH}, PY_VER: '3.12', CUDA_VER: '12.9.0', LOCAL_CTK: '1', GPU: ${gpu}, DRIVER: 'latest' }
- { ARCH: ${ARCH}, PY_VER: '3.13', CUDA_VER: '11.8.0', LOCAL_CTK: '0', GPU: ${gpu}, DRIVER: 'earliest' }
- { ARCH: ${ARCH}, PY_VER: '3.13', CUDA_VER: '11.8.0', LOCAL_CTK: '1', GPU: ${gpu}, DRIVER: 'latest' }
- { ARCH: ${ARCH}, PY_VER: '3.13', CUDA_VER: '12.0.1', LOCAL_CTK: '1', GPU: ${gpu}, DRIVER: 'latest' }
- { ARCH: ${ARCH}, PY_VER: '3.13', CUDA_VER: '12.9.0', LOCAL_CTK: '0', GPU: ${gpu}, DRIVER: 'latest' }
- { ARCH: ${ARCH}, PY_VER: '3.13', CUDA_VER: '12.9.0', LOCAL_CTK: '1', GPU: ${gpu}, DRIVER: 'latest' }
${special_runner}
"

# Use the nightly matrix for branch tests
MATRIX_TYPE="${BUILD_TYPE}"
if [[ "${MATRIX_TYPE}" == "branch" ]]; then
MATRIX_TYPE="nightly"
fi
export MATRIX_TYPE
TEST_MATRIX=$(yq -n 'env(MATRICES) | .[strenv(MATRIX_TYPE)]')
export TEST_MATRIX

# Read base matrix from JSON file for the specific architecture
TEST_MATRIX=$(jq --arg arch "$ARCH" --arg matrix_type "$MATRIX_TYPE" '
.linux[$matrix_type] |
map(select(.ARCH == $arch))
' ci/test-matrix.json)

# Add special runner for amd64 if applicable
if [[ "${ARCH}" == "amd64" ]]; then
SPECIAL_RUNNERS=$(jq '
.linux.special_runners.amd64
' ci/test-matrix.json)
TEST_MATRIX=$(jq --argjson special "$SPECIAL_RUNNERS" '. + $special' <<< "$TEST_MATRIX")
fi

MATRIX="$(
yq -n -o json 'env(TEST_MATRIX)' | \
jq -c '${{ inputs.matrix_filter }} | if (. | length) > 0 then {include: .} else "Error: Empty matrix\n" | halt_error(1) end'
jq -c '${{ inputs.matrix_filter }} | if (. | length) > 0 then {include: .} else "Error: Empty matrix\n" | halt_error(1) end' <<< "$TEST_MATRIX"
)"

echo "MATRIX=${MATRIX}" | tee --append "${GITHUB_OUTPUT}"
Expand Down
43 changes: 21 additions & 22 deletions .github/workflows/test-wheel-windows.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,11 @@ jobs:
outputs:
MATRIX: ${{ steps.compute-matrix.outputs.MATRIX }}
steps:
- name: Checkout ${{ github.event.repository.name }}
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
fetch-depth: 1

- name: Validate Test Type
run: |
if [[ "$BUILD_TYPE" != "pull-request" ]] && [[ "$BUILD_TYPE" != "nightly" ]] && [[ "$BUILD_TYPE" != "branch" ]]; then
Expand All @@ -41,47 +46,36 @@ jobs:
- name: Compute Python Test Matrix
id: compute-matrix
run: |
# Please keep the matrices sorted in ascending order by the following:
#
# [PY_VER, CUDA_VER, LOCAL_CTK]
#
export MATRICES="
pull-request:
- { ARCH: ${ARCH}, PY_VER: '3.12', CUDA_VER: '12.9.0', LOCAL_CTK: '0' }
- { ARCH: ${ARCH}, PY_VER: '3.12', CUDA_VER: '12.9.0', LOCAL_CTK: '1' }
- { ARCH: ${ARCH}, PY_VER: '3.13', CUDA_VER: '13.0.0', LOCAL_CTK: '0' }
- { ARCH: ${ARCH}, PY_VER: '3.13', CUDA_VER: '13.0.0', LOCAL_CTK: '1' }
nightly:
- { ARCH: ${ARCH}, PY_VER: '3.12', CUDA_VER: '11.8.0', LOCAL_CTK: '0' }
- { ARCH: ${ARCH}, PY_VER: '3.12', CUDA_VER: '11.8.0', LOCAL_CTK: '1' }
- { ARCH: ${ARCH}, PY_VER: '3.12', CUDA_VER: '12.9.0', LOCAL_CTK: '0' }
- { ARCH: ${ARCH}, PY_VER: '3.12', CUDA_VER: '12.9.0', LOCAL_CTK: '1' }
"

# Use the nightly matrix for branch tests
MATRIX_TYPE="${BUILD_TYPE}"
if [[ "${MATRIX_TYPE}" == "branch" ]]; then
MATRIX_TYPE="nightly"
fi
export MATRIX_TYPE
TEST_MATRIX=$(yq -n 'env(MATRICES) | .[strenv(MATRIX_TYPE)]')
export TEST_MATRIX

# Read base matrix from JSON file for the specific architecture
TEST_MATRIX=$(jq --arg arch "$ARCH" --arg matrix_type "$MATRIX_TYPE" '
.windows[$matrix_type] |
map(select(.ARCH == $arch))
' ci/test-matrix.json)

MATRIX="$(
yq -n -o json 'env(TEST_MATRIX)' | \
jq -c '${{ inputs.matrix_filter }} | if (. | length) > 0 then {include: .} else "Error: Empty matrix\n" | halt_error(1) end'
jq -c '${{ inputs.matrix_filter }} | if (. | length) > 0 then {include: .} else "Error: Empty matrix\n" | halt_error(1) end' <<< "$TEST_MATRIX"
)"

echo "MATRIX=${MATRIX}" | tee --append "${GITHUB_OUTPUT}"

test:
# TODO: switch to this once the self-hosted runners are ready
# name: py${{ matrix.PY_VER }}, ${{ matrix.CUDA_VER }}, ${{ (matrix.LOCAL_CTK == '1' && 'local') || 'wheels' }}, GPU ${{ matrix.GPU }}
name: py${{ matrix.PY_VER }}, ${{ matrix.CUDA_VER }}, ${{ (matrix.LOCAL_CTK == '1' && 'local') || 'wheels' }}
# The build stage could fail but we want the CI to keep moving.
needs: compute-matrix
strategy:
fail-fast: false
matrix: ${{ fromJSON(needs.compute-matrix.outputs.MATRIX) }}
if: ${{ github.repository_owner == 'nvidia' && !cancelled() }}
# TODO: switch to self-hosted runners once they are ready
# runs-on: "windows-${{ matrix.ARCH }}-gpu-${{ matrix.GPU }}-${{ matrix.DRIVER }}-1"
runs-on: 'cuda-python-windows-gpu-github'
steps:
- name: Checkout ${{ github.event.repository.name }}
Expand All @@ -90,6 +84,9 @@ jobs:
fetch-depth: 0

# TODO: use setup-proxy-cache once we have self-hosted Windows runners
# - name: Setup proxy cache
# uses: nv-gha-runners/setup-proxy-cache@main
# continue-on-error: true

- name: Update driver
run: |
Expand All @@ -98,6 +95,7 @@ jobs:
- name: Ensure GPU is working
run: nvidia-smi

# TODO: remove this block once self-hosted runners are ready
- name: Install Git for Windows
# the GPU runner image does not have Git Bash pre-installed...
env:
Expand Down Expand Up @@ -143,6 +141,7 @@ jobs:
name: ${{ env.CUDA_BINDINGS_ARTIFACT_NAME }}
path: ${{ env.CUDA_BINDINGS_ARTIFACTS_DIR }}

# TODO: remove this block once self-hosted runners are ready
- name: Install gh cli
# the GPU runner image does not have gh pre-installed...
env:
Expand Down
Loading