Skip to content

config: Support ~/.aws/config parsing#5378

Merged
efiop merged 3 commits into
treeverse:masterfrom
isidentical:s3-config-parser
Feb 4, 2021
Merged

config: Support ~/.aws/config parsing#5378
efiop merged 3 commits into
treeverse:masterfrom
isidentical:s3-config-parser

Conversation

@isidentical
Copy link
Copy Markdown
Contributor

@isidentical isidentical commented Feb 1, 2021

Thank you for the contribution - we'll try to review it as soon as possible. 🙏

resolves #4359

Comment thread dvc/config_schema.py
@isidentical isidentical marked this pull request as ready for review February 2, 2021 07:19
@isidentical isidentical requested a review from a team February 2, 2021 07:25
Copy link
Copy Markdown
Contributor

@pared pared left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@isidentical isidentical changed the title [WIP] config: Support ~/.aws/config parsing config: Support ~/.aws/config parsing Feb 3, 2021
Comment thread dvc/tree/s3.py Outdated
Comment thread dvc/utils/conversions.py
@@ -0,0 +1,24 @@
# https://github.com/aws/aws-cli/blob/5aa599949f60b6af554fd5714d7161aa272716f7/awscli/customizations/s3/utils.py
Copy link
Copy Markdown
Contributor

@efiop efiop Feb 4, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this whole file is awscli-specific, we could move this stuff to dvc/tree/s3.py or create something like dvc/tree/s3/utils.py and it put it there. This piece is quite small, so could go with the former, no problem.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think keeping here is OK (even though this is a pre-emptive assumption) I believe this utility might come in handy in other places since it does a very general job. The awscli-specific part here is the handling of IEC suffixes, though it doesn't differ much and something we can ignore.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, let's keep it then 👍

@efiop efiop merged commit e7cc776 into treeverse:master Feb 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support additional configuration parameters for S3 transfer

4 participants