cmd: copy edits for import-url ref. by utkarshsingh99 · Pull Request #1494 · treeverse/dvc.org

utkarshsingh99 · 2020-06-24T05:24:43Z

❗ Please read the guidelines in the Contributing to the Documentation list if you make any substantial changes to the documentation or JS engine.
🐛 Please make sure to mention Fix #issue (if applicable) in the description of the PR. This causes GitHub to close it automatically when the PR is merged.
Please choose to allow us to edit your branch when creating the PR.

Thank you for the contribution - we'll try to review it as soon as possible. 🙏

~~Partially address #1469~~

jorgeorpinel · 2020-06-24T06:29:41Z

You could add the "This" -> "These" update here 😉

utkarshsingh99 · 2020-06-26T15:27:27Z

Fix #1492

utkarshsingh99 · 2020-06-27T07:09:14Z

@jorgeorpinel I found import-url.md to be the best place for now to differentiate file hashes since all different remote locations are talked about in detail here including their requirements (such as strong ETag, etc) in DVC.
Since import.md (import command) just focuses on the DVC repos or Git repos, they will only have an md5 field, so I guess it isn't worth talking about different file hashes there.
I have added my comments for a few extra changes that I made. Let me know if they are not needed and I will scrap them out. ✌️

utkarshsingh99 · 2020-06-27T07:15:25Z


-You may want to get out of and remove the `example-get-started/` directory after
-trying this example (especially if trying out the following one).
-


I believe this statement is irrelevant here since anyone could arrive at this page without going through the User Guide/Getting-Started and it does not hold a direct info about the import-url command.

I don't think you followed the entire examples section first. That would need to be done to determine this. Feel free to do so and submit a different PR if you find this unnecessary, but it's out of scope for this PR. Please roll back here. Thanks

Alright.
But do you think it should be removed?

I don't think you followed the entire examples section first.

Actually, since this is in command reference, I thought it would be good to keep this file independent of the Getting Started module

I don't think it should be removed. But like I mentioned, I would need to follow the entire examples section first to decide.

would be good to keep this file independent of the Getting Started

It is. Again, I don't think you read the whole examples section 🙂

Please roll back.

Oh okay. Now I get it. It's just related to the example above.
My apologies 😄

jorgeorpinel · 2020-07-02T05:10:53Z

Partially address #1469

Please say specifically which part of that issue is addressed in the description of this PR (you can just copy the entire import/import-url checkbox here, and mark it).

Fix #1492

FYI special keywords only work on the PR description. Not needed here anyway because I already closed that issue as fixed.

I found import-url.md to be the best place for now to differentiate file hashes since all different remote locations are talked about

I see that import only has md5 in its examples, indeed. What about "add (external dep)" and "run -d (external dep)"? (The other checkboxes in ##1469) Did you check them? You don't have to for this PR, I'm just curious.

skshetry · 2020-07-02T05:33:28Z

+The `deps` field in the generated `.dvc` file contains a hash value along with
+the `path` to track changes in the remote file/directory. The hash type depends
+on the type of remote location (protocol) being tracked. For instance,
+files/directories tracked from HTTP(s)/S3/Azure/GS remote locations will have an


I made a mistake with the comment. GS uses md5. Sorry about that.

Okay, I'll change it in the DVC Files and Directories page in the next commit ✌️

No worries @skshetry, glad you caught it though! Thanks for the followup. I updated #1469 and fixed the dvc.lock doc in another PR. But this brings some questions... I'll ping you somewhere else.

~~@utkarshsingh99 please update the paragraph above accordingly...~~

utkarshsingh99 · 2020-07-02T07:16:34Z

What about "add (external dep)" and "run -d (external dep)"? (The other checkboxes in ##1469) Did you check them?

There's one mention of adding external files in dvc add but it's linking already to the External Dependencies page. Perhaps we can add a link to hash values' info here: dvc add example?

per #1494 (review)

jorgeorpinel · 2020-07-04T19:58:20Z

Nah actually add is a special case. Those aren't deps, they're outs. Don't worry about that for now.

But what about dvc run -d? I assume you didn't check it and that's fine. Let's just focus on addressing the remaining feedback up here ☝️ please, so we can merge this.

utkarshsingh99 · 2020-07-05T14:34:38Z

Removed this

For instance, files/directories tracked from HTTP(s)/S3/Azure remote locations will have an
ETag value in the deps field, while
files/directories tracked from GS/SSH/local locations will have an md5 hash
value. Files from HDFS remote locations use a checksum key in the deps
field.

Added:

See dvc.lock file for more info on hash types for specific remote location types.

Just adding this here since I got a bit confused with 2 reviews here: #1494(comment) and #1494(another comment).
Anyway, let me know if there's something wrong in this change and I'll make the changes immediately. ✌️

jorgeorpinel · 2020-07-06T07:10:26Z

+The `deps` field in the generated `.dvc` file contains a hash value along with
+the `path` to track changes in the remote file/directory. The hash type depends
+on the type of remote location (protocol) being tracked. See
+[`dvc.lock` file](/doc/user-guide/dvc-files-and-directories#dvclock-file) for
+info on hash types for specific remote location types.
+


Yeah this whole paragraph is probably unnecessary but thanks. The ref. is already too long TBH. Users can find out about .dvc file specs in its dedicated guide, that's not the main purpose for a command ref.

Okay, so where should I link dvc.lock file ?

Why did you decide to edit this command reference if there's no existing places that relate to dvc.lock? I'm going to just merge this with the few typo fixed but I'll have to uncheck that box in the original index so this can be re-evaluated.

I actually thought that we needed to add some reference to the hash values in each file or describe where it was needed based on the original issue. But does this mean that there is no place we can add a reference to dvc.lock or provide info about different types of file hashes in import-url?

does this mean that there is no place we can add a reference to dvc.lock or provide info about different types of file hashes in import-url?

Idk, can you answer that question? 🙂 Let's go back to analyzing the checkboxes of #1469 and please comment there if any one seems unaplicable.

jorgeorpinel · 2020-07-06T07:18:21Z

confused with 2 reviews

Ah yeah, it's the first one: remove whole paragraph. Thanks

update content/docs/command-reference/add.md import-url.md

d011d32

utkarshsingh99 mentioned this pull request Jun 24, 2020

docs: differentiate types of file hashes #1469

Closed

3 tasks

jorgeorpinel reviewed Jun 24, 2020

View reviewed changes

Comment thread content/docs/command-reference/add.md Outdated

jorgeorpinel reviewed Jun 24, 2020

View reviewed changes

Comment thread content/docs/command-reference/import-url.md Outdated

Update content/docs/command-reference/import-url.md

f58218d

utkarshsingh99 added 2 commits June 26, 2020 20:55

update docs/start/data-pipelines.md

f8597a1

Merge branch 'patch12' of github.com:utkarshsingh99/dvc.org into patch12

2c97b0c

utkarshsingh99 added 4 commits June 27, 2020 11:25

statement not needed, confusing: cmd-ref/import-url.md

738d034

hash value expl, undo in cmd-ref:add.md

93d2264

update cmd-ref/import-url.md

88db141

merge conflicts fix: cmd-ref/import-url.md

b29a3de

utkarshsingh99 commented Jun 27, 2020

View reviewed changes

Comment thread content/docs/command-reference/import-url.md Outdated

utkarshsingh99 commented Jun 27, 2020

View reviewed changes

Comment thread content/docs/command-reference/import-url.md Outdated

utkarshsingh99 commented Jun 27, 2020

View reviewed changes

jorgeorpinel mentioned this pull request Jun 27, 2020

docs - get started: git contents inaccurate #1492

Closed

utkarshsingh99 marked this pull request as ready for review June 27, 2020 10:04

utkarshsingh99 requested a review from jorgeorpinel June 27, 2020 10:19

This comment has been minimized.

Sign in to view

jorgeorpinel mentioned this pull request Jul 2, 2020

update docs/start/experiments.md #1511

Merged

3 tasks

skshetry reviewed Jul 2, 2020

View reviewed changes

jorgeorpinel reviewed Jul 2, 2020

View reviewed changes

Comment thread content/docs/command-reference/import-url.md Outdated

This comment has been minimized.

Sign in to view

utkarshsingh99 added 3 commits July 2, 2020 11:41

rolling back changes

518f80d

semantic changes - review

300ab7d

DVC File changes

ebe47dd

jorgeorpinel reviewed Jul 4, 2020

View reviewed changes

Comment thread content/docs/user-guide/dvc-files-and-directories.md Outdated

jorgeorpinel added a commit that referenced this pull request Jul 4, 2020

guide: update types of checksums in dvc.lock doc

3acbc8d

per #1494 (review)

jorgeorpinel mentioned this pull request Jul 4, 2020

guide: update types of checksums in dvc.lock doc #1527

Merged

jorgeorpinel changed the title ~~added info about different file hashes~~ cmd: added info about different file hashes to import-url Jul 4, 2020

weekly-digest Bot mentioned this pull request Jul 5, 2020

Weekly Digest (28 June, 2020 - 5 July, 2020) #1529

Closed

update docs/cmd-ref/import-url.md

1448063

rolling back changes: docs/user-guide/dvc-files-and-directories.md

a31deed

utkarshsingh99 requested a review from jorgeorpinel July 5, 2020 14:40

jorgeorpinel reviewed Jul 6, 2020

View reviewed changes

Comment thread content/docs/command-reference/import-url.md Outdated

jorgeorpinel reviewed Jul 6, 2020

View reviewed changes

update docs/cmd-ref/import-url.md

635d360

jorgeorpinel reviewed Jul 7, 2020

View reviewed changes

Comment thread content/docs/command-reference/import-url.md Outdated

Update content/docs/command-reference/import-url.md

249e03f

jorgeorpinel changed the title ~~cmd: added info about different file hashes to import-url~~ cmd: copy edits for import-url ref. Jul 7, 2020

jorgeorpinel approved these changes Jul 7, 2020

View reviewed changes

jorgeorpinel merged commit 4426244 into treeverse:master Jul 7, 2020

weekly-digest Bot mentioned this pull request Jul 12, 2020

Weekly Digest (5 July, 2020 - 12 July, 2020) #1563

Closed


		You may want to get out of and remove the `example-get-started/` directory after
		trying this example (especially if trying out the following one).

Conversation

utkarshsingh99 commented Jun 24, 2020 • edited by jorgeorpinel Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jorgeorpinel commented Jun 24, 2020

Uh oh!

utkarshsingh99 commented Jun 26, 2020

Uh oh!

utkarshsingh99 commented Jun 27, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

utkarshsingh99 Jun 27, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

jorgeorpinel commented Jul 2, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jorgeorpinel Jul 4, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

This comment has been minimized.

utkarshsingh99 commented Jul 2, 2020

Uh oh!

Uh oh!

jorgeorpinel commented Jul 4, 2020

Uh oh!

utkarshsingh99 commented Jul 5, 2020

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jorgeorpinel commented Jul 6, 2020

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

utkarshsingh99 commented Jun 24, 2020 •

edited by jorgeorpinel

Loading

utkarshsingh99 commented Jun 27, 2020 •

edited

Loading

utkarshsingh99 Jun 27, 2020 •

edited

Loading

jorgeorpinel Jul 4, 2020 •

edited

Loading