Add in-repo Complement test to sanity check Synapse version matches git checkout by MadLittleMods · Pull Request #19476 · element-hq/synapse

MadLittleMods · 2026-02-18T02:19:27Z

Add in-repo Complement test to sanity check Synapse version matches git checkout

This way we actually detect problems like #19475 as they happen instead of being invisible until something breaks.

Sanity check that Complement is testing against your code changes (whether it be local or from the PR in CI).

COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh --in-repo -run TestSynapseVersion

Dev notes

Synapse version string generated by https://github.com/matrix-org/matrix-python-common/blob/4084b21af839c50f775447d02ca4f1854e2e6191/src/matrix_common/versionstring.py

poetry run python -c 'from synapse.util import SYNAPSE_VERSION; print(SYNAPSE_VERSION)'

https://docs.docker.com/build/metadata/attestations/slsa-provenance/

Pull Request Checklist

Pull request is based on the develop branch
Pull request includes a changelog file. The entry should:
- Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from EventStore to EventWorkerStore.".
- Use markdown where necessary, mostly for code blocks.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry.
Code style is correct (run the linters)

…it checkout

This reverts commit 5293622.

…site-packages/synapse/SYNAPSE_VERSION` See #19476 (comment)

MadLittleMods · 2026-03-02T16:39:37Z

@@ -707,28 +707,28 @@ jobs:
          cache-dependency-path: complement/go.sum


This PR is purposely not up to date with the develop at the moment so I can still reproduce the problem from #19475

lol, trying to get this new test to fail in CI but it looks like GitHub Current runner version: '2.332.0' is back to using Docker Client/Server 28.0.4 (source) which doesn't have the problem.

Versus the previous GitHub Current runner version: '2.331.0' we saw in #19475 which was using Docker Client/Server 29.1.5 (source)

Looks like this was reverted in actions/runner-images#13708

reivilibre

Seems like a good idea given it bit us, but I guess we need a workaround for the egg info. (Maybe rm it opportunistically in the complement.sh script? Might be our best option)

reivilibre · 2026-03-03T12:09:10Z

+
+	// Then run our actual command
+	cmd := exec.Command(commandPieces[0], commandPieces[1:]...)
+	output, err := cmd.Output()


Consider cmd.CombinedOutput() to get stderr as well. Currently I think stderr output would go uncaptured?

I think that's expected as we just want the machine-readable/pipeable values from stdout

Although I don't think it matters in this case with the command we're using.

Ideally, we'd print everything when the command fails 👍

reivilibre · 2026-03-03T12:09:10Z

+		checkoutSynapseVersion := runCommand(
+			t,
+			[]string{
+				"poetry",


I guess it's not necessarily a given that people will have poetry if our uv plans pick up any momentum at all. However I'm also happy to leave this easy fix for when it's actually a problem.

I would expect this to be updated to whatever our developer flow is ⏩

reivilibre · 2026-03-03T12:09:11Z

+# available in the Docker image for Synapse to generate from.
+ARG SYNAPSE_VERSION_STRING
+# Pass it through to Synapse as an environment variable.
+ENV SYNAPSE_VERSION_STRING=${SYNAPSE_VERSION_STRING}


(had to look up that this is actually optional, seems like it is)

See #19476 (comment)

MadLittleMods · 2026-03-05T03:06:10Z

+        # We don't run `gotestfmt` for the sanity check step as it's only one test so
+        # there is nothing to organize. And the extra step is more noise to click
+        # through and grok. If this step fails, it will be obvious still.
+


Before I update this branch with the latest develop, I want to see it fail.

But CI won't run for some reason. I assume that I have a syntax error in my update here but how do I find it? I thought GitHub would tell me somewhere

Testing out CI by itself in #19532

#19532 has the exact same CI but it runs there 🤔

I feel like this has something to do with the conflict/outdated but I don't want to update for the reason above.

Seems to run when I manually trigger the workflow (from this page) for this branch, https://github.com/element-hq/synapse/actions/runs/22733335657

And we can actually see an error for the complement job ❌:

synapse/scripts-dev/complement.sh: line 199: poetry: command not found Error: Process completed with exit code 127.

(got the sanity check test running, see commits below)

Still don't know why CI isn't triggering on its own in this PR.

I feel like this has something to do with the conflict/outdated but I don't want to update for the reason above.

Found some evidence of this in https://github.com/orgs/community/discussions/11265

And usually when I naturally come across this situation, I just update the branch and things work. But in this case, I didn't want to immediately do it so we just end up stuck with no feedback

Yeah, it's both intuitive and not at the same time. The CI runs against the result of merging the PR into the target branch (good so that you get the most accurate thing being tested). They should really add a warning when there are conflicts present that this blocks CI...

Usage: command [options] [arguments] Options: -h, --help Display help for the given command. When no command is given display help for the list command. -q, --quiet Do not output any message. -V, --version Display this application version. --ansi Force ANSI output. --no-ansi Disable ANSI output. -n, --no-interaction Do not ask any interactive question. --no-plugins Disables plugins. --no-cache Disables Poetry source caches. -P, --project=PROJECT Specify another path as the project root. All command-line arguments will be resolved relative to the current working directory. -C, --directory=DIRECTORY The working directory for the Poetry command (defaults to the current working directory). All command-line arguments will be resolved relative to the given directory. -v|vv|vvv, --verbose Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug. Available commands: about Shows information about Poetry. add Adds a new dependency to pyproject.toml and installs it. build Builds a package, as a tarball and a wheel by default. check Validates the content of the pyproject.toml file and its consistency with the poetry.lock file. config Manages configuration settings. export Exports the lock file to alternative formats. help Displays help for a command. init Creates a basic pyproject.toml file in the current directory. install Installs the project dependencies. list Lists commands. lock Locks the project dependencies. new Creates a new Python project at <path>. publish Publishes a package to a remote repository. remove Removes a package from the project dependencies. run Runs a command in the appropriate environment. search Searches for packages on remote repositories. show Shows information about packages. sync Update the project's environment according to the lockfile. update Update the dependencies as according to the pyproject.toml file. version Shows the version of the project or bumps it when a valid bump rule is provided. cache cache clear Clears a Poetry cache by name. cache list List Poetry's caches. debug debug info Shows debug information. debug resolve Debugs dependency resolution. debug tags Shows compatible tags for your project's current active environment. env env activate Print the command to activate a virtual environment. env info Displays information about the current environment. env list Lists all virtualenvs associated with the current project. env remove Remove virtual environments associated with the project. env use Activates or creates a new virtualenv for the current project. python python install Install the specified Python version from the Python Standalone Builds project. (experimental feature) python list Shows Python versions available for this environment. (experimental feature) python remove Remove the specified Python version if managed by Poetry. (experimental feature) self self add Add additional packages to Poetry's runtime environment. self install Install locked packages (incl. addons) required by this Poetry installation. self lock Lock the Poetry installation's system requirements. self remove Remove additional packages from Poetry's runtime environment. self show Show packages from Poetry's runtime environment. self show plugins Shows information about the currently installed plugins. self sync Sync Poetry's own environment according to the locked packages (incl. addons) required by this Poetry installation. self update Updates Poetry to the latest version. source source add Add source configuration for project. source remove Remove source configured for the project. source show Show information about sources configured for the project. available

Alternatively, we could avoid the JSON

MadLittleMods · 2026-03-05T20:28:04Z

+        # real-time.
+        run: |
+          set -o pipefail
+          COMPLEMENT_DIR=`pwd`/complement synapse/scripts-dev/complement.sh --in-repo -p 1 -json -run 'TestSynapseVersion/Synapse_version_matches_current_git_checkout' 2>&1 | tee /tmp/gotest-sanity-check-complement.log


With go test, I wish there was a way to enforce that some test is actually run with this -run '...' filter.

Since golang/go#64500 was rejected, best we could do is grep over the output for no tests to run. But it also gets slightly more tricky as no tests to run will be printed for the other packages that don't have a matching test.

Probably just move on without this ⏩

As a final stop-gap, this test will also be run as part of the normal --in-repo run below. So at-least things won't be able to accidentally pass if this test name changes from under us. Running it first here, just gives us the proper feedback on the first source of wrong-ness.

MadLittleMods · 2026-03-05T20:47:50Z

@@ -699,6 +699,13 @@ jobs:
          toolchain: ${{ env.RUST_VERSION }}
      - uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2



We could also copy these changes to .github/workflows/latest_deps.yml and .github/workflows/twisted_trunk.yml where we also run Complement tests.

We already have the --in-repo variant there so the sanity check test will run as part of that. We just won't have the initial feedback of this first run like we have here in .github/workflows/tests.yml

Can easily copy across if you poke. Ideally, all of this boilerplate would be shared as @anoadragon453 suggested.

Making a re-usable Complement workflow in #19533

Conflicts: scripts-dev/complement.sh

reivilibre

(The commit message on 296ba32 is funny, did you accidentally let poetry get expanded by your shell? :)))

Anyway this seems good to me, thanks for putting in the effort!

reivilibre · 2026-03-11T11:25:45Z

+        # We don't run `gotestfmt` for the sanity check step as it's only one test so
+        # there is nothing to organize. And the extra step is more noise to click
+        # through and grok. If this step fails, it will be obvious still.
+


Yeah, it's both intuitive and not at the same time. The CI runs against the result of merging the PR into the target branch (good so that you get the most accurate thing being tested). They should really add a warning when there are conflicts present that this blocks CI...

MadLittleMods · 2026-03-11T14:53:26Z

@@ -705,6 +705,13 @@ jobs:
          toolchain: ${{ env.RUST_VERSION }}
      - uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2



(The commit message on 296ba32 is funny, did you accidentally let poetry get expanded by your shell? :)))

-- #19476 (review)

Yep. Really easy to do when anything in backticks is executed and you like to format with markdown. Pushed it anyway ⏩

MadLittleMods · 2026-03-11T20:31:32Z

@reivilibre Thanks for the review and help figuring out the Python packaging puzzles 😵

…omplement.sh` (#19578) ❌ `Build and push complement image`, https://github.com/element-hq/synapse/actions/runs/23176317296/job/67339146082 ``` scripts-dev/complement.sh: line 227: poetry: command not found ``` Follow-up to #19523 This regressed in #19476 ### Testing strategy 1. Visit https://github.com/element-hq/synapse/actions/workflows/push_complement_image.yml 1. **Run workflow**: - **Use workflow from:** `madlittlemods/fix-complement-push-image-ci-job-poetry` - **Branch:** `develop` 1. Wait for CI to run and pass ✅

This PR contains the following updates: | Package | Update | Change | |---|---|---| | [element-hq/synapse](https://github.com/element-hq/synapse) | minor | `v1.149.1` → `v1.150.0` | --- ### Release Notes <details> <summary>element-hq/synapse (element-hq/synapse)</summary> ### [`v1.150.0`](https://github.com/element-hq/synapse/releases/tag/v1.150.0) [Compare Source](element-hq/synapse@v1.149.1...v1.150.0) ### Synapse 1.150.0 (2026-03-24) No significant changes since 1.150.0rc1. ### Synapse 1.150.0rc1 (2026-03-17) #### Features - Add experimental support for the [MSC4370](matrix-org/matrix-spec-proposals#4370) Federation API `GET /extremities` endpoint. ([#19314](element-hq/synapse#19314)) - [MSC4140: Cancellable delayed events](matrix-org/matrix-spec-proposals#4140): When persisting a delayed event to the timeline, include its `delay_id` in the event's `unsigned` section in `/sync` responses to the event sender. ([#19479](element-hq/synapse#19479)) - Expose [MSC4354 Sticky Events](matrix-org/matrix-spec-proposals#4354) over the legacy (v3) /sync API. ([#19487](element-hq/synapse#19487)) - When Matrix Authentication Service (MAS) integration is enabled, allow MAS to set the user locked status in Synapse. ([#19554](element-hq/synapse#19554)) #### Bugfixes - Fix `Build and push complement image` CI job pointing to non-existent image. ([#19523](element-hq/synapse#19523)) - Fix a bug introduced in v1.26.0 that caused deactivated, erased users to not be removed from the user directory. ([#19542](element-hq/synapse#19542)) #### Improved Documentation - In the Admin API documentation, always express path parameters as `/<param>` instead of as `/$param`. ([#19307](element-hq/synapse#19307)) - Update docs to clarify `outbound_federation_restricted_to` can also be used with the [Secure Border Gateway (SBG)](https://element.io/en/server-suite/secure-border-gateways). ([#19517](element-hq/synapse#19517)) - Unify Complement developer docs. ([#19518](element-hq/synapse#19518)) #### Internal Changes - Put membership updates in a background resumable task when changing the avatar or the display name. ([#19311](element-hq/synapse#19311)) - Add in-repo Complement test to sanity check Synapse version matches git checkout (testing what we think we are). ([#19476](element-hq/synapse#19476)) - Migrate `dev` dependencies to [PEP 735](https://peps.python.org/pep-0735/) dependency groups. ([#19490](element-hq/synapse#19490)) - Remove the optional `systemd-python` dependency and the `systemd` extra on the `synapse` package. ([#19491](element-hq/synapse#19491)) - Avoid re-computing the event ID when cloning events. ([#19527](element-hq/synapse#19527)) - Allow caching of the `/versions` and `/auth_metadata` public endpoints. ([#19530](element-hq/synapse#19530)) - Add a few labels to the number groupings in the `Processed request` logs. ([#19548](element-hq/synapse#19548)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).  Reviewed-on: https://gitea.alexlebens.dev/alexlebens/infrastructure/pulls/5040 Co-authored-by: Renovate Bot <renovate-bot@alexlebens.net> Co-committed-by: Renovate Bot <renovate-bot@alexlebens.net>

Add in-repo Complement test to sanity check Synapse version matches g…

e1587cc

…it checkout

MadLittleMods added the A-Testing label Feb 18, 2026

MadLittleMods mentioned this pull request Feb 18, 2026

Fix Complement CI not running against the code from our PRs (remote images being chosen over local) #19475

Merged

5 tasks

MadLittleMods added 8 commits February 18, 2026 10:29

Better comments

59df0d6

Add tests for parseSynapseVersionString

49d2d27

Working parseSynapseVersionString

ce1ae57

Run git commands

5189af9

getVersionFromPyproject

77ed60f

Better error message

991ab59

Fix dirty checkout command

639402b

Add changelog

29029e5

MadLittleMods commented Feb 18, 2026

View reviewed changes

Comment thread complement/tests/synapse_version_check_test.go

MadLittleMods commented Feb 18, 2026

View reviewed changes

Comment thread complement/go.mod

MadLittleMods added 4 commits February 18, 2026 16:47

Try adding .git data

5293622

Revert "Try adding .git data"

bc2c676

This reverts commit 5293622.

Pretty print structs we compare

aef8a88

Start of SYNAPSE_VERSION

b58aab6

MadLittleMods commented Feb 19, 2026

View reviewed changes

Comment thread complement/tests/synapse_version_check_test.go Outdated

MadLittleMods commented Feb 19, 2026

View reviewed changes

Comment thread pyproject.toml Outdated

MadLittleMods commented Feb 19, 2026

View reviewed changes

Comment thread pyproject.toml Outdated

MadLittleMods added 6 commits February 20, 2026 09:55

Fix SYNAPSE_VERSION not appearing under `/usr/local/lib/python3.13/…

6ce2ca2

…site-packages/synapse/SYNAPSE_VERSION` See #19476 (comment)

Read from file

5d1a25f

Remove SYNAPSE_VERSION file strategy

4b33f3f

Read from environment variable

e8c5bad

Update docstring

944611c

Fix lints

ddfbbf8

MadLittleMods commented Feb 20, 2026

View reviewed changes

Comment thread scripts-dev/complement.sh

MadLittleMods commented Feb 20, 2026

View reviewed changes

Comment thread complement/tests/synapse_version_check_test.go Outdated

Simplify test

b7bd4be

MadLittleMods commented Feb 20, 2026

View reviewed changes

Comment thread synapse/util/__init__.py Outdated

MadLittleMods commented Mar 2, 2026

View reviewed changes

reivilibre self-requested a review March 3, 2026 11:32

reivilibre requested changes Mar 3, 2026

View reviewed changes

MadLittleMods added 5 commits March 4, 2026 20:13

PR's -> PRs

107066c

See #19476 (comment)

Remove double be typo

0b5cb91

See #19476 (comment)

Fix overridden typo

3f7f4f5

See #19476 (comment)

Remove egg-info to get accurate Synapse version string

b1a651e

See #19476 (comment)

Update CI

f2c92ec

MadLittleMods commented Mar 5, 2026

View reviewed changes

MadLittleMods added a commit that referenced this pull request Mar 5, 2026

Try out CI for #19476

4e3e5bb

MadLittleMods mentioned this pull request Mar 5, 2026

Try out CI for #19476 #19532

Closed

3 tasks

MadLittleMods added 6 commits March 5, 2026 12:19

Kick CI again

7bdc821

Point to Synapse checkout

957d5c3

Move closer to Synapse step

c28cd6c

Formatting would be nice because JSON is hard to read

7b199ab

Alternatively, we could avoid the JSON

Test is in repo

5dd4805

MadLittleMods commented Mar 5, 2026

View reviewed changes

Merge branch 'develop' into madlittlemods/sanity-check-complement-image

4109f09

Conflicts: scripts-dev/complement.sh

MadLittleMods requested a review from reivilibre March 5, 2026 23:14

reivilibre approved these changes Mar 11, 2026

View reviewed changes

MadLittleMods commented Mar 11, 2026

View reviewed changes

Merge branch 'develop' into madlittlemods/sanity-check-complement-image

c5c5465

MadLittleMods merged commit e300018 into develop Mar 11, 2026
78 of 81 checks passed

MadLittleMods deleted the madlittlemods/sanity-check-complement-image branch March 11, 2026 20:30

MadLittleMods mentioned this pull request Mar 17, 2026

Fix Build and push complement image CI job not having Poetry for complement.sh #19578

Merged

3 tasks

		@@ -707,28 +707,28 @@ jobs:
		cache-dependency-path: complement/go.sum

		@@ -699,6 +699,13 @@ jobs:
		toolchain: ${{ env.RUST_VERSION }}
		- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2

		@@ -705,6 +705,13 @@ jobs:
		toolchain: ${{ env.RUST_VERSION }}
		- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2

Conversation

MadLittleMods commented Feb 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Dev notes

Pull Request Checklist

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MadLittleMods Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

reivilibre left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

MadLittleMods Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MadLittleMods Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MadLittleMods Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

reivilibre left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

MadLittleMods commented Mar 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

MadLittleMods commented Feb 18, 2026 •

edited

Loading

MadLittleMods Mar 5, 2026 •

edited

Loading

MadLittleMods Mar 5, 2026 •

edited

Loading

MadLittleMods Mar 5, 2026 •

edited

Loading

MadLittleMods Mar 5, 2026 •

edited

Loading