Problem Statement
When using Alibaba Cloud OSS as the AGFS S3 backend, add_resource always fails with a 400 error during temporary directory cleanup after file processing:
failed to delete source directory: failed to delete objects:
operation error S3: DeleteObjects, StatusCode: 400,
api error MissingArgument: Missing Some Required Arguments.
The missing argument is Content-MD5. OSS requires this header for the DeleteObjects (POST /?delete) API, but AWS SDK v2 does not send it by default. This
makes the S3 backend completely unusable with Alibaba Cloud OSS.
Proposed Solution
Add a disable_batch_delete boolean option to S3Config. When set to true, the backend uses sequential single-object DeleteObject calls instead of batch
DeleteObjects. Single-object delete does not require Content-MD5, making it compatible with OSS.
The option defaults to false, so existing users are not affected.
Example configuration:
"s3": {
"endpoint": "http://s3.oss-cn-beijing.aliyuncs.com",
"bucket": "your-bucket",
"region": "cn-beijing",
"disable_batch_delete": true
}
Use Case
Using Alibaba Cloud OSS (s3.oss-{region}.aliyuncs.com) as the AGFS S3 storage backend. OSS is a widely used object storage service that follows the
S3-compatible API with minor differences — one being the mandatory Content-MD5 header for batch delete operations.
Problem Statement
When using Alibaba Cloud OSS as the AGFS S3 backend,
add_resourcealways fails with a 400 error during temporary directory cleanup after file processing:The missing argument is
Content-MD5. OSS requires this header for theDeleteObjects(POST /?delete) API, but AWS SDK v2 does not send it by default. Thismakes the S3 backend completely unusable with Alibaba Cloud OSS.
Proposed Solution
Add a
disable_batch_deleteboolean option toS3Config. When set totrue, the backend uses sequential single-objectDeleteObjectcalls instead of batchDeleteObjects. Single-object delete does not requireContent-MD5, making it compatible with OSS.The option defaults to
false, so existing users are not affected.Example configuration:
Use Case
Using Alibaba Cloud OSS (
s3.oss-{region}.aliyuncs.com) as the AGFS S3 storage backend. OSS is a widely used object storage service that follows theS3-compatible API with minor differences — one being the mandatory
Content-MD5header for batch delete operations.