For now the only way to pass credentials to S3 is using credentials file and the env var. This is very inconvenient when trying to use dvc as a library. For now I hacked it like:
def pull(repo):
# In memory config update to avoid validation
repo.config["core"]["remote"] = "vdefault"
repo.config["remote"]["vdefault"] = {"url": ..., "key_id": ..., "key_secret": ...}
fix_s3()
repo.pull()
from funcy import once, monkey, cached_property, wrap_prop # noqa
@once
def fix_s3():
import threading
from dvc.remote.s3 import S3RemoteTree
@monkey(S3RemoteTree, "s3")
@wrap_prop(threading.Lock())
@cached_property
def s3(self):
import boto3
session = boto3.session.Session(
profile_name=self.profile,
region_name=self.region,
# Pass these two from config to session
aws_access_key_id=self.config.get("key_id"),
aws_secret_access_key=self.config.get("key_secret"),
)
return session.client("s3", endpoint_url=self.endpoint_url, use_ssl=self.use_ssl)Would be nice to have it inside dvc.remote.S3RemoteTree.
For now the only way to pass credentials to S3 is using credentials file and the env var. This is very inconvenient when trying to use dvc as a library. For now I hacked it like:
Would be nice to have it inside
dvc.remote.S3RemoteTree.