Skip to content

feat(gcs): allow setting a token directly#4978

Merged
Xuanwo merged 13 commits intoapache:mainfrom
jdockerty:feat/support-gcs-token
Aug 13, 2024
Merged

feat(gcs): allow setting a token directly#4978
Xuanwo merged 13 commits intoapache:mainfrom
jdockerty:feat/support-gcs-token

Conversation

@jdockerty
Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes #4876

Rationale for this change

An OAuth2 token cannot be set directly at the moment, instead other mechanisms are required such as the credential or credential_path.

What changes are included in this PR?

Inclusion of a token within the GcsConfig and corresponding methods so that a bearer token is set within the signed requests to GCP.

Are there any user-facing changes?

Direct token being available as a option for authentication

@jdockerty jdockerty requested a review from Xuanwo as a code owner August 7, 2024 15:53
@github-actions github-actions Bot added the releases-note/feat The PR implements a new feature or has a title that begins with "feat" label Aug 7, 2024
Comment thread core/src/services/gcs/backend.rs Outdated
Comment thread core/src/services/gcs/backend.rs Outdated
Comment thread core/src/services/gcs/core.rs Outdated
@Xuanwo
Copy link
Copy Markdown
Member

Xuanwo commented Aug 8, 2024

Hi, #4979 has been merged, we can continue this one now.

Comment thread core/src/services/gcs/core.rs Outdated
Comment thread core/src/services/gcs/backend.rs Outdated
signer,
token_loader,
token: self.config.token,
scope: self.config.scope,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

scope will be DEFAULT_GCS_SCOPE if not set.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I've altered this now.

I skimmed over that it was being set already with a default, I'll combine this with the direct String usage too. Much appreciated 👍

Comment thread core/src/services/gcs/core.rs Outdated

impl GcsCore {
async fn load_token(&self) -> Result<Option<GoogleToken>> {
match (&self.token, &self.scope) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

scope must be valid, we can store as String directly.

Comment thread core/src/services/gcs/core.rs Outdated
}

pub async fn sign<T>(&self, req: &mut Request<T>) -> Result<()> {
if let Some(token) = &self.token {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need to change the logic here since load_token has already handled this.

Copy link
Copy Markdown
Contributor Author

@jdockerty jdockerty Aug 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense 👍

Do I understand rightly that the same is also true for sign_query? I.e. my addition here should also be removed.

if let Some(token) = &self.token {
req.headers_mut().remove(HOST);
let header_value = format!("Bearer {}", token);
req.headers_mut()
.insert(header::AUTHORIZATION, header_value.parse().unwrap());
return Ok(());
}

I've added ☝️, but the sign_query is handled through load_credential and not load_token

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, this part is a bit complex. Let me elaborate.


GCS's token-based authorization doesn't support signed queries. Signed queries involve credentials to build presigned URLs, similar to AWS S3. We can't build such presigned URLs using a token. So if users only set a token but not credentials, the signed query won't work.

However, I believe the changes here are the same, and we don't need to alter code sign_xxx. Just let load_xxx handle it.

Copy link
Copy Markdown
Member

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thanks!

@Xuanwo Xuanwo merged commit eb7d430 into apache:main Aug 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

releases-note/feat The PR implements a new feature or has a title that begins with "feat"

Projects

None yet

Development

Successfully merging this pull request may close these issues.

gcs: Should allow setting token directly

2 participants