S3 support for data storage

[pinkmanbox]

In a project I am working on, I want data in an S3 bucket to be available to pull locally, while maintaining the folder structure and the data as it is in the bucket. This is because I need to perform AWS operations on that data as well. But since I also need that data for local operations (and any changes on them), dvc control would be beneficial.

Essentially, S3 being the GitHub for the data, while dvc being the git for the data.

I apologize if this feature already exists, and if so, could someone please direct me on how to achieve this ?

Thanks.

[pared]

Related discussion and some context:
https://discordapp.com/channels/485586884165107732/563406153334128681/677407602979766275

[efiop]

@pinkmanbox Sounds like you might be looking for dvc get/import commands. Please take a look at https://dvc.org/doc/use-cases/data-registries . TLDR; is that dvc remote is not a dvc repo, it requires git to store metadata, so you could address your files by name through your git repo and dvc will automatically download them from your s3 bucket. Let us know what you think.

[efiop]

Closing as stale for now. Please feel free to reopen.