[Bug]: Unable to rename files residing in external storage (S3)

[datenangebot]

⚠️ This issue respects the following points: ⚠️

• This is a bug, not a question or a configuration/webserver/proxy issue.
• This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
• Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
• I agree to follow Nextcloud's Code of Conduct.

Bug description

After upgrading to 27.1.3, we discovered that it is not possible anymore to rename files that reside in external S3 storage.

Exception from the server:
{"Exception":"Aws\\Exception\\MultipartUploadException","Message":"An exception occurred while uploading parts to a multipart upload. The following parts had errors:\n- Part 1: Error executing \"UploadPartCopy\" on \"https://???/test2.odt?partNumber=1&uploadId=2~Ifrvg95tAnlyFxa75hfrysf0stGe9Wb\"; AWS HTTP error: Client error: `PUT https://???/test2.odt?partNumber=1&uploadId=2~Ifrvg95tAnlyFxa75hfrysf0stGe9Wb` resulted in a `404 Not Found` response:\n<?xml version=\"1.0\" encoding=\"UTF-8\"?><Error><Code>NoSuchKey</Code><BucketName>???</BucketName><Reque (truncated...)\n NoSuchKey (client):  - <?xml version=\"1.0\" encoding=\"UTF-8\"?><Error><Code>NoSuchKey</Code><BucketName>???</BucketName><RequestId>tx00000e4cce185faa47f8f-006565dcde-f59e029-default</RequestId><HostId>???</HostId></Error>\n",

Steps to reproduce

Install NC27.1.3 with S3 storage.

Expected behavior

Can rename

Installation method

None

Nextcloud Server version

27

Operating system

None

PHP engine version

None

Web server

None

Database engine version

None

Is this bug present after an update or on a fresh install?

None

Are you using the Nextcloud Server Encryption module?

None

What user-backends are you using?

• Default user-backend (database)
• LDAP/ Active Directory
• SSO - SAML
• Other

Configuration report

No response

List of activated Apps

No response

Nextcloud Signing status

No response

Nextcloud Logs

No response

Additional info

Seems to be related to: #40577
If we revert this patch, it works fine.

[mickenordin]

I went through and disabled all apps one by one to confirm that it was not related to guzzle being vendored twice like I have seen for some other folks. And reverting the patch indeed fixes the issue. In our case Ceph is used to provide S3 storage.

This is the exact patch I used to revert the multi part upload on 27.1.4.2 if someone is interested

diff --git a/apps/files_external/lib/Lib/Storage/AmazonS3.php b/apps/files_external/lib/Lib/Storage/AmazonS3.php
index 8e2a6b8ea12..5619b7be8dd 100644
--- a/apps/files_external/lib/Lib/Storage/AmazonS3.php
+++ b/apps/files_external/lib/Lib/Storage/AmazonS3.php
@@ -582,7 +582,10 @@ class AmazonS3 extends \OC\Files\Storage\Common {
 
 		if ($isFile === true || $this->is_file($source)) {
 			try {
-				$this->copyObject($source, $target, [
+ 				$this->getConnection()->copyObject([
+ 					'Bucket' => $this->bucket,
+ 					'Key' => $this->cleanKey($target),
+ 					'CopySource' => S3Client::encodeKey($this->bucket . '/' . $source),
 					'StorageClass' => $this->storageClass,
 				]);
 				$this->testTimeout();
diff --git a/lib/private/Files/ObjectStore/S3ObjectTrait.php b/lib/private/Files/ObjectStore/S3ObjectTrait.php
index 217e1a1a2ff..7328287acee 100644
--- a/lib/private/Files/ObjectStore/S3ObjectTrait.php
+++ b/lib/private/Files/ObjectStore/S3ObjectTrait.php
@@ -189,23 +189,9 @@ trait S3ObjectTrait {
 	public function objectExists($urn) {
 		return $this->getConnection()->doesObjectExist($this->bucket, $urn, $this->getSSECParameters());
 	}
-
-	public function copyObject($from, $to, array $options = []) {
-		$sourceMetadata = $this->getConnection()->headObject([
-			'Bucket' => $this->getBucket(),
-			'Key' => $from,
-		] + $this->getSSECParameters());
-
-		$copy = new MultipartCopy($this->getConnection(), [
-			"source_bucket" => $this->getBucket(),
-			"source_key" => $from
-		], array_merge([
-			"bucket" => $this->getBucket(),
-			"key" => $to,
-			"acl" => "private",
-			"params" => $this->getSSECParameters() + $this->getSSECParameters(true),
-			"source_metadata" => $sourceMetadata
-		], $options));
-		$copy->copy();
+ 	public function copyObject($from, $to) {
+ 		$this->getConnection()->copy($this->getBucket(), $from, $this->getBucket(), $to, 'private', [
+ 			'params' => $this->getSSECParameters() + $this->getSSECParameters(true)
+ 		]);
 	}
 }

[joshtrichards]

Fixed in #41914 (Primary Storage) + #43060 (External Storage)

You will need to toggle the flag on the External Storage to disable multipartcopy.