I'm developing against s3s-fs and using OpenDAL to read the data.
It looks like listing currently expects an <ETag> to be set, despite this being marked as optional in the Amazon docs (https://docs.aws.amazon.com/AmazonS3/latest/API/API_Object.html).
The following diff seems to work around the problem.
I'll try and raise a proper PR for this tiny change, but I'd appreciate feedback in the meantime.
(I see @Xuanwo in the commit history for this part of the code, so tagging you - I hope you don't mind!).
diff --git a/core/src/services/s3/pager.rs b/core/src/services/s3/pager.rs
index 3c5b15ed0..30646da8a 100644
--- a/core/src/services/s3/pager.rs
+++ b/core/src/services/s3/pager.rs
@@ -126,8 +126,10 @@ impl oio::Page for S3Pager {
let mut meta = Metadata::new(EntryMode::FILE);
- meta.set_etag(&object.etag);
- meta.set_content_md5(object.etag.trim_matches('"'));
+ if let Some(etag) = &object.etag {
+ meta.set_etag(&etag);
+ meta.set_content_md5(etag.trim_matches('"'));
+ }
meta.set_content_length(object.size);
// object.last_modified provides more precious time that contains
@@ -168,7 +170,7 @@ struct OutputContent {
size: u64,
last_modified: String,
#[serde(rename = "ETag")]
- etag: String,
+ etag: Option<String>,
}
#[derive(Default, Debug, Eq, PartialEq, Deserialize)]
I'm developing against
s3s-fsand using OpenDAL to read the data.It looks like listing currently expects an
<ETag>to be set, despite this being marked as optional in the Amazon docs (https://docs.aws.amazon.com/AmazonS3/latest/API/API_Object.html).The following diff seems to work around the problem.
I'll try and raise a proper PR for this tiny change, but I'd appreciate feedback in the meantime.
(I see @Xuanwo in the commit history for this part of the code, so tagging you - I hope you don't mind!).