This repository was archived by the owner on Apr 26, 2024. It is now read-only.
-
-
Notifications
You must be signed in to change notification settings - Fork 2.1k
Make e2e backup versions numeric in the DB #4113
Merged
Merged
Changes from all commits
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
563f9b6
Make e2e backup versions numeric in the DB
dbkr 64fa557
Try & make it work on postgres
dbkr 4eacf0f
news fragment
dbkr 2f0f911
Convert version back to a string
dbkr 12941f5
Cast bacjup version to int when querying
dbkr e0934ac
Cast to int here too
dbkr d3fa619
Remove unnecessary str()
dbkr 4f93abd
add docs
dbkr d44dea0
pep8
dbkr bca3b91
Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backup_ve…
dbkr File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Fix e2e key backup with more than 9 backup versions |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| /* Copyright 2018 New Vector Ltd | ||
| * | ||
| * Licensed under the Apache License, Version 2.0 (the "License"); | ||
| * you may not use this file except in compliance with the License. | ||
| * You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
|
|
||
| /* Change version column to an integer so we can do MAX() sensibly | ||
| */ | ||
| CREATE TABLE e2e_room_keys_versions_new ( | ||
| user_id TEXT NOT NULL, | ||
| version BIGINT NOT NULL, | ||
| algorithm TEXT NOT NULL, | ||
| auth_data TEXT NOT NULL, | ||
| deleted SMALLINT DEFAULT 0 NOT NULL | ||
| ); | ||
|
|
||
| INSERT INTO e2e_room_keys_versions_new | ||
| SELECT user_id, CAST(version as BIGINT), algorithm, auth_data, deleted FROM e2e_room_keys_versions; | ||
|
|
||
| DROP TABLE e2e_room_keys_versions; | ||
| ALTER TABLE e2e_room_keys_versions_new RENAME TO e2e_room_keys_versions; | ||
|
|
||
| CREATE UNIQUE INDEX e2e_room_keys_versions_idx ON e2e_room_keys_versions(user_id, version); | ||
|
|
||
| /* Change e2e_rooms_keys to match | ||
| */ | ||
| CREATE TABLE e2e_room_keys_new ( | ||
| user_id TEXT NOT NULL, | ||
| room_id TEXT NOT NULL, | ||
| session_id TEXT NOT NULL, | ||
| version BIGINT NOT NULL, | ||
| first_message_index INT, | ||
| forwarded_count INT, | ||
| is_verified BOOLEAN, | ||
| session_data TEXT NOT NULL | ||
| ); | ||
|
|
||
| INSERT INTO e2e_room_keys_new | ||
| SELECT user_id, room_id, session_id, CAST(version as BIGINT), first_message_index, forwarded_count, is_verified, session_data FROM e2e_room_keys; | ||
|
|
||
| DROP TABLE e2e_room_keys; | ||
| ALTER TABLE e2e_room_keys_new RENAME TO e2e_room_keys; | ||
|
|
||
| CREATE UNIQUE INDEX e2e_room_keys_idx ON e2e_room_keys(user_id, room_id, session_id); |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we not just keep this as an int?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's a string in the API, ie. when json encoded for the client it needs to be
"2"rather than2, so we need to stringify it at some point. I don't really mind whether we model them as int throughout the whole of synapse & convert to str at the last minute for marshalling or just keep them as ints at the database layer.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a reason to then not have them as strings in the database layer?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It currently does a MAX() at the db layer to get the largest version, then increments it in python, which breaks when the column type is a string because "10" < "2" which is the bug this fixes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why does the API have it as a string when JSON has numbers?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
#e2e-dev:matrix.org has the API / spec discussion: this one was: https://matrix.to/#/!uewiilduiDRfPomIha:matrix.org/$15397878202555374QpBig:matrix.org