ref: clarify about obfuscated remote file structure (same as cache)

[antonkulaga]

From my experience of moving the team to DVC there were two major understanding problems with understanding of key DVC concepts:

• when you do a gdrive and other dvc remotes (with exception of dvc import) they have folders/files in a dvc-specific format, you cannot just put regular input files to gdrive and say they are remote (unless you are using dvc import)
• dvc run is more about creating stages than actually running, if one just wants to run she should use dvc repro instead

Despite reading dvc tutorials and docs many teammates and students were still confused by this. Maybe there are ways to emphasize it somehow so people who read not attentively will not confuse?

[efiop]

Transferred to the docs repo.

[jorgeorpinel]

Hei @antonkulaga thanks for the feedback

dvc remotes... have folders/files in a dvc-specific format, you cannot just put regular input files to gdrive and say they are remote (unless you are using dvc import)

This is something we can definitely improve in our docs, I think. Probably around sync commands (push/pull/etc) and remote refs. and maybe mention that remotes use the same https://dvc.org/doc/user-guide/dvc-files-and-directories#structure-of-the-cache-directory (and link there from the aforementioned pages).

We'll keep this issue open to be addressed when possible, but feel free to contribute, that would be amazing! (Just a draft PR is enough, we can polish the writing, add links, etc.)

dvc run is more about creating stages than actually running

Yep, it used to be for running before 1.0 but it's transitioned into a helper to define stages. We're considering renaming it to something like dvc stage add I think (cc @efiop that's more of a discussion for the core repo?) and leaving the executing part for repro exclusively.

For now in docs we already emphasize that it's a helper command so nothing actionable here at the moment, I think. Unless you can point us to specific docs to look at. Thanks

[sarthakforwet]

@jorgeorpinel In push.md, we have the sentence "dvc push uploads data from the cache to remote storage."
Doesn't this sentence provide the argument that remote storage would have data in a dvc-specific format?

Also, can you please mention some references for me to learn about this issue in a better way? Thanks!

[jorgeorpinel]

Hey @sarthakforwet all the info about the cache dir structure is in the link I shared above (https://dvc.org/doc/user-guide/dvc-files-and-directories#structure-of-the-cache-directory).

dvc push uploads data from the cache to remote storage

It's a hint, yes. But it doesn't necessarily imply the cache structure is kept on the remote. It could be much more explicit — not sure which docs to change though, it needs some looking around and reading the existing ones.

p.s. I see you are currently working on #1852, thanks BTW! I recommend you focus on that for now.