Overwriting existing packages in backend storage can lead to caching issues

[mbearup]

If an existing package is re-added to pulp, the default behavior will overwrite the existing file in backing storage. This is typically fine.

• If using Azure Blobstore, the timestamp of the blob is updated (Last-Modified time and ETag).
• Conversely, some CDN's (notably Azure Front Door) use Last-Modified Time as a signal that a file in origin has updated.
• This can lead to poor cache behavior, and in some cases, incomplete downloads as the CDN attempts to resolve disparate content.
• If we set AZURE_OVERWRITE_FILES to false this partially mitigates the issue (Last-Modified/ETag are unmodified). However, this results in duplicate copies written to storage (with a suffix to differentiate from the original).
• We should have an option that does "nothing" if the uploaded file already exists (don't overwrite, and don't write a new copy).

[daviddavis]

I was able to reproduce this with a pulp dev environment using minio. Steps to reproduce:

pulp rpm repository create --name test
pulp rpm distribution create --name test --repository test --base-path test
pulp rpm content upload --file bear-4.1-1.noarch.rpm
pulp rpm repository content add --repository test --package-href /pulp/api/v3/content/rpm/packages/018e6776-e768-7f2e-bff5-8aeb5bf6e43b/  
pulp rpm publication create --repository test

# now check the Last-Modified header
curl -sL --head http://localhost:24816/pulp/content/test/Packages/b/bear-4.1-1.noarch.rpm | grep "Last-Modified"

# upload the package again
pulp rpm content upload --file bear-4.1-1.noarch.rpm > /dev/null

# check the Last-Modified header again
curl -sL --head http://localhost:24816/pulp/content/test/Packages/b/bear-4.1-1.noarch.rpm | grep "Last-Modified"

Here's the output I see:

$ curl -sL --head http://localhost:24816/pulp/content/test/Packages/b/bear-4.1-1.noarch.rpm | grep "Last-Modified"
Last-Modified: Fri, 22 Mar 2024 18:40:29 GMT           

$ pulp rpm content upload --file bear-4.1-1.noarch.rpm > /dev/null
Started background task /pulp/api/v3/tasks/018e6782-e5f5-7b25-9e6f-d1e240170130/
.Done.

$ curl -sL --head http://localhost:24816/pulp/content/test/Packages/b/bear-4.1-1.noarch.rpm | grep "Last-Modified"
Last-Modified: Fri, 22 Mar 2024 18:53:35 GMT                                                               

So the Last Modified header changes when a package is uploaded. I definitely think that reuploading a package shouldn't change its modified timestamp for various reasons.

Like Azure, there is a AWS_S3_FILE_OVERWRITE option (defaults to True) but setting it to False makes django-storages create new files in storage with random chars as a suffix which is problematic.

[daviddavis]

Adding a check here before save to try to fetch any existing artifact seems to fix the problem. I think what's happening is that django-storages updates the file in storage regardless of whether the artifact already exists or not.

[dkliban]

@daviddavis do you want to open a PR?