fix: support AWS_REQUEST_PAYER=requester#669
fix: support AWS_REQUEST_PAYER=requester#669ljstrnadiii wants to merge 4 commits intoapache:mainfrom
AWS_REQUEST_PAYER=requester#669Conversation
"requester" for AWS_REQUEST_PAYER env varAWS_REQUEST_PAYER=requester
src/aws/builder.rs
Outdated
| } | ||
| AmazonS3ConfigKey::RequestPayer => { | ||
| self.request_payer = ConfigValue::Deferred(value.into()) | ||
| // Support the standard AWS value "requester" as a boolean true |
There was a problem hiding this comment.
the parsing logic should be part of the value type, like this:
arrow-rs-object-store/src/aws/builder.rs
Lines 1330 to 1365 in bdcac43
Now from the user PoV this doesn't make a difference, but using this pattern makes the code easier to read and maintain (no surprising "why is this value handled differently and where is the code that does this out of the sudden?" surprises)
There was a problem hiding this comment.
Thanks for the guidance. Will make this change today.
There was a problem hiding this comment.
@crepererum Putting this as part of the impl Parse for bool
arrow-rs-object-store/src/config.rs
Lines 74 to 86 in bdcac43
means that any boolean flag is enabled by the string "requester", right?
From my point of view if we allow this case, it seems perhaps better to isolate it to the AWS_REQUEST_PAYER code path? 🤷♂️
There was a problem hiding this comment.
I also went back and forth on this, but @kylebarron has a good point. I'll wait for followup from @crepererum before I do anything here.
There was a problem hiding this comment.
I would probably use a new-type wrapper for that, like:
// derive whatever you need, but basically this is just a `bool`
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash]
struct RequesterPayer(bool);
impl Parse for RequesterPayer {
fn parse(v: &str) -> Result<Self> {
if v.eq_ignore_ascii_case("requester") {
Ok(Self(true))
} else {
Ok(Self(bool::parse(v)?))
}
}
}
// you may want to implement a few convenience traits
impl From<RequesterPayer> for bool {...}
impl AsRef<bool> for RequesterPayer {...}
impl Deref for RequesterPayer {...}There was a problem hiding this comment.
The newtype wrapper sounds great.
crepererum
left a comment
There was a problem hiding this comment.
Small nitpicks, but overall this looks pretty good already.
|
|
||
| #[test] | ||
| fn test_request_payer_config() { | ||
| let s3 = AmazonS3Builder::new() |
There was a problem hiding this comment.
Can I get a case-insensitive test case too, e.g. Requester or REQUESTER?
| Some(bucket.strip_suffix("--x-s3")?.rsplit_once("--")?.1) | ||
| } | ||
|
|
||
| #[derive(Debug, Default, Clone, Copy, PartialEq, Eq, Hash)] |
There was a problem hiding this comment.
| #[derive(Debug, Default, Clone, Copy, PartialEq, Eq, Hash)] | |
| /// Captures `AWS_REQUEST_PAYER`. | |
| /// | |
| /// Parses either as `"requester"` (case-insensitive) meaning `true`, or as a [`bool`] (i.e. `"true"`, `"1"`, `"no"`, etc.). | |
| #[derive(Debug, Default, Clone, Copy, PartialEq, Eq, Hash)] |
Feel free to rephrase, but I think leaving some documentation might be nice.
Which issue does this PR close?
Closes #668
Rationale for this change
Avoid AWS_REQUEST_PAYER env var collision.
What changes are included in this PR?
Resolves 'requester' to True for requester_pays
Are there any user-facing changes?
No.