RFC #112: Media versioning

[thibaudcolas]

This RFC is a broad review of the requirements of Media versioning on the Wagtail roadmap. The main goal at this point is to clarify the user needs / requirements we want to cater for, and likely technical hurdles. There are a lot of gaps left in analysing how best to implement those requirements, in particular the suitability of RevisionMixin.

Highlights:

• Focus on core versioning capabilities, while also mentioning related common requirements.
• A lot of questions left open for others to reflect on which ones definitely need answering sooner than later.

[tomusher]

This would be a welcome addition!

A couple of thoughts as I was reading through:

It's anecdotal, but on the sites I've worked with, users tend to treat Documents/Images like a filesystem - to replace a file they'll just upload a new one with a version in the name.

This is likely not ideal for various reasons, but I wonder if there is some consideration around the UX here to encourage users to edit existing Documents rather than just adding new ones.

---

As mentioned, many sites using custom storage backends to store media in blob storage like S3. These can often be configured with versioning enabled.

Would it make sense to offload management of media version history to the storage backend? i.e. historic versions could be retrieved directly from the S3 API rather than being stored in the database.

[thibaudcolas]

thanks @tomusher! I’ve added those two as open questions in the doc. For UX - my intuition is yes it’d be better to encourage users to edit existing items, like we discourage uploading of duplicates. But I’m not sure if we should go for slight tweaks or if there is a need to be heavy-handed with this.

For versioning with S3, I’m honestly not sure -- I don’t know how much additional value there is in working with backend-specific versioning APIs if we needed to support storage backends with no versioning anyway. If we did, it would probably be outside of core, say in wagtail-storages?

[kaedroho]

Another approach could be to store the file contents and metadata separately, for example: https://github.com/kaedroho/django-react-cms/blob/main/server/djangopress/files/models.py.

The file contents (Blob model) is immutable, and the metadata (File model) is changeable and could be versioned. There is a ForeignKey from File to the current Blob, the value of this ForeignKey could be stored in the revision.

You'd have to make a decision about when you delete Blobs that are no longer used but are referenced in a revision, perhaps could set a expiry date when the last reference to the Blob is removed and delete it then (after that, you can't revert back to that version).

Another advantage of a separate model for file contents is that you could implement features like large, resumable uploading in one place and reuse for all different types of media

[thibaudcolas]

Other feedback on use cases from @pvetch:

• audit trail data - we have some clients for whom this would be important (and it could also be useful to be including this data on page (document last update on X/Y/Z by ABC).
• I guess I can see a case for locking an asset, to prevent a key document being replaced other than by users with elevated privileges.
• Scheduled publishing I guess could be interesting in a situation where you want to include a link to a document from a page that is already live and have the document appear as "not yet available" until the point when the document itself is set to be allowed to be visible... it would be a heavy edge case
• Workflows I can imagine being useful for image QA or something like that but in an environment where you need that kind of process, you'd almost certainly have a digital asset management system anyway

And link to ResourceSpace DAM version control docs.

[thibaudcolas]

Further feedback from @zerolab from clients’ need for private media:

• Visibility of unpublished information really crucial ("need to know")
• Those requirements are possible to implement for pages and snippets. Trickier for media
  • Current approach: media URLs changing based on whether the associated content (page?) has been published or not. Custom serve view with extra checks. Updating file visibility in S3, using signed URLs to access private docs. Implementation link: private_media app

This kind of functionality feels important to consider as part of versioning reqs. As far as user expectations I’d imagine this is desirable for all sorts of sites, though there are lots of good tech reasons why it’s not worth it for most.

[kaedroho]

Visibility of unpublished information really crucial ("need to know")

Just to clarify, is this about preventing access to media that hasn't been used on a published page?

[thibaudcolas]

yep