Skip to content

Commit 5f25ac8

Browse files
authored
Add parallel testing demo (#15)
* Add parallel testing demo * Update README * Cleanup
1 parent 5d79db0 commit 5f25ac8

File tree

10 files changed

+212
-39
lines changed

10 files changed

+212
-39
lines changed

.github/workflows/basic_uv.yml

Lines changed: 3 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
name: "Basic (uv)"
1+
name: Basic (uv)
22

33
on:
44
push:
@@ -18,14 +18,10 @@ jobs:
1818
with:
1919
enable-cache: true
2020

21-
- name: Setup python
22-
run: |
23-
uv python install
24-
uv run -- python --version
25-
2621
- name: Install python dependencies
2722
run: |
28-
uv sync --locked
23+
uv python install; echo
24+
uv sync --locked; echo
2925
uv tree
3026
3127
- name: Run test

.github/workflows/cache.yml

Lines changed: 5 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -22,18 +22,15 @@ jobs:
2222
2323
- name: Show dates.txt
2424
run: |
25-
if [[ -f "dates.txt" ]]; then
25+
if [[ -f 'dates.txt' ]]; then
2626
cat dates.txt
2727
else
28-
echo "Initial run. File not found. Creating..."
28+
echo 'Initial run. File not found. Creating...'
2929
touch dates.txt
3030
fi
3131
32-
- name: Add today to top of dates.txt
32+
- name: Add today and take top 5
3333
run: |
34-
date -u "+%Y-%m-%d %H:%M:%S UTC" > today.txt
35-
cat today.txt dates.txt | head -n 5 > new_dates.txt
34+
date -u '+%Y-%m-%d %H:%M:%S UTC' > today.txt
35+
cat today.txt dates.txt | head -n 5 | tee new_dates.txt
3636
cp new_dates.txt dates.txt
37-
38-
- name: Show updated dates.txt
39-
run: cat dates.txt

.github/workflows/env_var_pass.yml

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -14,9 +14,9 @@ jobs:
1414
steps:
1515
- name: Set environment variable (takes effect in subsequent steps)
1616
run: |
17-
echo "FOO=abc" >> "${GITHUB_ENV}"
18-
echo "BAR=123" >> "${GITHUB_ENV}"
19-
echo "No effect in the current step."
17+
echo 'FOO=abc' >> "${GITHUB_ENV}"
18+
echo 'BAR=123' >> "${GITHUB_ENV}"
19+
echo 'No effect in the current step.'
2020
echo "FOO is '${FOO}'"
2121
echo "BAR is '${BAR}'"
2222
@@ -42,7 +42,7 @@ jobs:
4242
id: pass
4343
run: |
4444
# Single-line using echo
45-
echo "GREETING=Hello, World!" >> "${GITHUB_OUTPUT}"
45+
echo 'GREETING=Hello, World!' >> "${GITHUB_OUTPUT}"
4646
4747
# Multi-line using heredoc
4848
{
@@ -62,5 +62,5 @@ jobs:
6262

6363
- name: Show RESPONSE
6464
run: |
65-
echo "RESPONSE is"
65+
echo 'RESPONSE is'
6666
echo '${{ needs.job2.outputs.RESPONSE }}'

.github/workflows/env_var_path.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ jobs:
1818
- name: Add PATH (takes effect in subsequent steps)
1919
run: |
2020
echo "${HOME}/imaginary" >> "${GITHUB_PATH}"
21-
echo "No effect in the current step."
21+
echo 'No effect in the current step.'
2222
echo "PATH is '${PATH}'"
2323
2424
- name: Show updated PATH

.github/workflows/env_var_write.yml

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -25,20 +25,20 @@ jobs:
2525
2626
- name: Overwrite months (takes effect in subsequent steps)
2727
run: |
28-
echo "JAN=01" >> "${GITHUB_ENV}"
29-
echo "FEB=02" >> "${GITHUB_ENV}"
30-
echo "MAR=03" >> "${GITHUB_ENV}"
31-
echo "No effect in the current step."
28+
echo 'JAN=01' >> "${GITHUB_ENV}"
29+
echo 'FEB=02' >> "${GITHUB_ENV}"
30+
echo 'MAR=03' >> "${GITHUB_ENV}"
31+
echo 'No effect in the current step.'
3232
echo "JAN is '${JAN}'"
3333
echo "FEB is '${FEB}'"
3434
echo "MAR is '${MAR}'"
3535
3636
- name: New months (takes effect in subsequent steps)
3737
run: |
38-
echo "APR=04" >> "${GITHUB_ENV}"
39-
echo "MAY=05" >> "${GITHUB_ENV}"
40-
echo "JUN=06" >> "${GITHUB_ENV}"
41-
echo "No effect in the current step."
38+
echo 'APR=04' >> "${GITHUB_ENV}"
39+
echo 'MAY=05' >> "${GITHUB_ENV}"
40+
echo 'JUN=06' >> "${GITHUB_ENV}"
41+
echo 'No effect in the current step.'
4242
echo "APR is '${APR}'"
4343
echo "MAY is '${MAY}'"
4444
echo "JUN is '${JUN}'"

.github/workflows/github_script.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,14 +13,14 @@ jobs:
1313

1414
steps:
1515
- name: Return string
16-
uses: actions/github-script@v7
1716
id: return-result
17+
uses: actions/github-script@v7
1818
with:
1919
result-encoding: string # Choices: string, json (Default: json)
2020
script: return "Hello, World!"
2121

2222
- name: Get string
23-
run: echo "${{ steps.return-result.outputs.result }}"
23+
run: echo '${{ steps.return-result.outputs.result }}'
2424

2525
- name: Show the first 10 fibonacci numbers
2626
uses: actions/github-script@v7

.github/workflows/homebrew.yml

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -13,17 +13,15 @@ jobs:
1313

1414
steps:
1515
- name: Check brew
16-
run: which brew || echo "Program not found"
16+
run: which brew || echo 'Program not found.'
1717

1818
- name: Check fortune
19-
run: which fortune || echo "Program not found"
19+
run: which fortune || echo 'Program not found.'
2020

2121
- uses: Homebrew/actions/setup-homebrew@master
2222

2323
- name: Install fortune
2424
run: brew install fortune
2525

2626
- name: Run fortune
27-
run: |
28-
which fortune
29-
fortune
27+
run: fortune

.github/workflows/parallel_dir.yml

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
name: Parallel (dir)
2+
3+
on:
4+
push:
5+
schedule:
6+
- cron: "0 0 1 * *" # Midnight every month (UTC)
7+
workflow_dispatch:
8+
9+
jobs:
10+
collect:
11+
name: Collect
12+
runs-on: ubuntu-latest
13+
outputs:
14+
directories: "${{ steps.build.outputs.TEST_DIRS }}"
15+
16+
steps:
17+
- uses: actions/checkout@v4
18+
19+
- uses: astral-sh/setup-uv@v6
20+
with:
21+
enable-cache: true
22+
23+
- name: Install python dependencies
24+
run: |
25+
uv python install; echo
26+
uv sync --locked; echo
27+
uv tree
28+
29+
- name: Collect tests
30+
run: |
31+
uv run -- pytest --collect-only --quiet | tee collected.tmp
32+
echo '::group::Delete the last two lines'
33+
sed -e '$d' collected.tmp | sed -e '$d' | tee collected.txt
34+
echo '::endgroup::'
35+
36+
- name: Build test matrix
37+
id: build
38+
shell: uv run -- python {0}
39+
run: |
40+
from os import environ
41+
from pathlib import Path
42+
43+
# Deduplicate directory paths.
44+
with open("collected.txt", encoding="utf-8") as lines:
45+
test_dirs = sorted({str(Path(line.partition("::")[0]).parent) for line in lines})
46+
47+
print(f"::notice ::Directories: {test_dirs}")
48+
with open(environ["GITHUB_OUTPUT"], "a", encoding="utf-8") as f:
49+
print(f"TEST_DIRS={test_dirs}", file=f)
50+
51+
test:
52+
name: Test
53+
runs-on: ubuntu-latest
54+
needs: collect
55+
56+
strategy:
57+
matrix:
58+
directory: ${{ fromJSON(needs.collect.outputs.directories) }}
59+
60+
steps:
61+
- uses: actions/checkout@v4
62+
63+
- uses: astral-sh/setup-uv@v6
64+
with:
65+
enable-cache: true
66+
67+
- name: Install python dependencies
68+
run: |
69+
uv python install; echo
70+
uv sync --locked; echo
71+
uv tree
72+
73+
- name: Set pytest norecursedirs
74+
# Prevent the test runner from recursing into subdirectories. This
75+
# avoids duplicate test runs if there are nested test directories.
76+
run: |
77+
echo 'norecursedirs = "*"' >> pyproject.toml
78+
echo '::notice ::Enabled pytest norecursedirs.'
79+
80+
- name: Run test
81+
# Quote the matrix argument because the path may contain spaces.
82+
run: uv run -- pytest '${{ matrix.directory }}'
Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
name: Parallel (file)
2+
3+
on:
4+
push:
5+
schedule:
6+
- cron: "0 0 1 * *" # Midnight every month (UTC)
7+
workflow_dispatch:
8+
9+
jobs:
10+
collect:
11+
name: Collect
12+
runs-on: ubuntu-latest
13+
outputs:
14+
groups: "${{ steps.build.outputs.TEST_GROUPS }}"
15+
16+
steps:
17+
- uses: actions/checkout@v4
18+
19+
- uses: astral-sh/setup-uv@v6
20+
with:
21+
enable-cache: true
22+
23+
- name: Install python dependencies
24+
run: |
25+
uv python install; echo
26+
uv sync --locked; echo
27+
uv tree
28+
29+
- name: Collect tests
30+
run: |
31+
uv run -- pytest --collect-only --quiet | tee collected.tmp
32+
echo '::group::Delete the last two lines'
33+
sed -e '$d' collected.tmp | sed -e '$d' | tee collected.txt
34+
echo '::endgroup::'
35+
36+
- name: Build test matrix
37+
id: build
38+
shell: uv run -- python {0}
39+
run: |
40+
from os import environ
41+
42+
# Deduplicate file paths.
43+
with open("collected.txt", encoding="utf-8") as lines:
44+
paths = sorted({line.partition("::")[0] for line in lines})
45+
46+
max_size = 3 # Max files per group
47+
test_groups = {"include": []}
48+
49+
# Slice the list of paths into sublists. We also take the precaution
50+
# to quote each path in case there are spaces within the path.
51+
for i, j in enumerate(range(0, len(paths), max_size), start=1):
52+
quoted = [f"'{path}'" for path in paths[j : j + max_size]]
53+
test_groups["include"].append({"name": f"Group {i}", "paths": " ".join(quoted)})
54+
55+
print(f"::notice ::Groups: {test_groups}")
56+
with open(environ["GITHUB_OUTPUT"], "a", encoding="utf-8") as f:
57+
print(f"TEST_GROUPS={test_groups}", file=f)
58+
59+
test:
60+
name: Test ${{ matrix.name }}
61+
runs-on: ubuntu-latest
62+
needs: collect
63+
64+
strategy:
65+
matrix: ${{ fromJSON(needs.collect.outputs.groups) }}
66+
67+
steps:
68+
- uses: actions/checkout@v4
69+
70+
- uses: astral-sh/setup-uv@v6
71+
with:
72+
enable-cache: true
73+
74+
- name: Install python dependencies
75+
run: |
76+
uv python install; echo
77+
uv sync --locked; echo
78+
uv tree
79+
80+
- name: Run test
81+
# Don't quote the matrix argument because it causes the shell
82+
# to treat the entire list of paths as a single argument.
83+
run: uv run -- pytest ${{ matrix.paths }}

README.md

Lines changed: 20 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -165,7 +165,7 @@ See the [workflow](.github/workflows/log_annotation.yml).
165165

166166
<summary>Define a matrix of different job configurations.</summary>
167167

168-
<br/>The matrix strategy lets easily target multiple operating systems or test multiple versions of a language.
168+
<br/>The matrix strategy helps you easily target multiple operating systems and language versions.
169169

170170
- Uses `actions/setup-node`
171171

@@ -187,6 +187,20 @@ See the [workflow](.github/workflows/mise.yml).
187187

188188
</details>
189189

190+
## parallel_*.yml
191+
192+
<details>
193+
194+
<summary>Parallel testing without any code changes or extra dependencies.</summary>
195+
196+
<br/>The matrix strategy can be used in a particular way to enable parallel testing for free. This means no code changes and no extra dependencies. This won't be a single runner using multiple cores, it's multiple runners each running its own set of tests. The idea is to identify where your tests are and then distrubute them across multiple machines. If your test runner supports parallel testing, you can use that in combination with this strategy to really go fast!
197+
198+
See the workflows:
199+
- [Directory-level parallel testing](.github/workflows/parallel_dir.yml)
200+
- [File-level parallel testing](.github/workflows/parallel_file.yml)
201+
202+
</details>
203+
190204
## References
191205

192206
- [GitHub Actions](https://docs.github.com/en/actions)
@@ -196,10 +210,13 @@ See the [workflow](.github/workflows/mise.yml).
196210
- [GitHub Actions: Adding a system path](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/workflow-commands-for-github-actions#adding-a-system-path)
197211
- [GitHub Actions: Caching dependencies to speed up workflows](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/caching-dependencies-to-speed-up-workflows)
198212
- [GitHub Actions: Events that trigger workflows](https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows)
199-
- [GitHub Actions: jobs.<job_id>.outputs](https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions#jobsjob_idoutputs)
200-
- [GitHub Actions: jobs.<job_id>.strategy.matrix](https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions#jobsjob_idstrategymatrix)
213+
- [Running variations of jobs in a workflow](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/running-variations-of-jobs-in-a-workflow)
201214
- [GitHub Actions: Setting an environment variable](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/workflow-commands-for-github-actions#setting-an-environment-variable)
215+
<br/><br/>
202216
- [GitHub Actions: Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions)
217+
- [GitHub Actions: env](https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions#env)
218+
- [GitHub Actions: jobs.<job_id>.outputs](https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions#jobsjob_idoutputs)
219+
- [GitHub Actions: jobs.<job_id>.strategy.matrix](https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions#jobsjob_idstrategymatrix)
203220
<br/><br/>
204221
- [actions/cache](https://github.com/actions/cache)
205222
- [actions/checkout](https://github.com/actions/checkout)

0 commit comments

Comments
 (0)