cmd-ref: document new gc behavior by skshetry · Pull Request #1023 · treeverse/dvc.org

skshetry · 2020-02-25T15:16:15Z

You may disregard these recommendations if you used the Edit on GitHub button from dvc.org to improve a doc in place.

❗ Please read the guidelines in the Contributing to the Documentation list if you make any substantial changes to the documentation or JS engine.

🐛 Please make sure to mention Fix #issue (if applicable) in the description of the PR. This causes GitHub to close it automatically when the PR is merged.

Please chose to allow us to edit your branch when creating the PR.

Thank you for the contribution - we'll try to review it as soon as possible. 🙏

Fixes #1013

shcheklein

looks great! the major thing is to align option description with other similar options

jorgeorpinel

Looking great in general, some precisions below. Not sure whether we're taking this one over (yet), please lmk @shcheklein, thanks

jorgeorpinel

Extracted from #1023 (comment)

One last improvement is to also update the description of -a and -T options. Something like:

- `-a`, `--all-branches` - keep cached objects referenced in all Git branches
  as well as in the workspace (implies `-w`). Useful for preserving data for ...

...

- `-T`, `--all-tags` - the same as `-a` above, but applies to Git tags as well
  as in the workspace (implies `-w`). Useful if tags are used to ...

Thanks

shcheklein · 2020-03-07T01:53:52Z

@skshetry @jorgeorpinel what is the status of it? @skshetry is still assigned - are you going to move it forward?

skshetry · 2020-03-07T02:24:33Z

@shcheklein Umm, yes. I'm trying to add --all-commits on this PR as well. I'll also have to address @jorgeorpinel reviews.

shcheklein · 2020-03-07T03:29:08Z

@skshetry sure! np. Just wanted to be sure that I don't block it like it happened last time.

shcheklein · 2020-03-15T02:40:42Z

@jorgeorpinel @skshetry ping! any progress on this?

skshetry · 2020-03-16T06:49:46Z

@shcheklein, sorry, I couldn't get to the docs. I have adjusted it now and addressed most of the suggestions. 🙂

Please take a look.

shcheklein · 2020-03-18T05:40:04Z

@skshetry @jorgeorpinel PTAL - https://dvc-landing-gc-flag-cha-stdoiq.herokuapp.com/doc/command-reference/gc - I've applied some edits to the doc

skshetry · 2020-03-18T06:59:42Z

-the local cache, which is typically located on the same machine as the project
-in question. This usually helps to free up disk space.
+This command deletes (garbage collects) data files or directories that exist in
+DVC cache but are no longer needed. With `--cloud` it also removes data in


but are no longer needed.

It'd be better to express this in any other way. I don't have a better suggestion, though than the following:

Suggested change

DVC cache but are no longer needed. With `--cloud` it also removes data in

DVC cache but are no longer in use. With `--cloud` it also removes data in

Good question, I think needed was there before. I don't have a very strong opinion ... may be even needed better because it is more precise here - I just keep stuff that I need. need is more general then use. Still, need to review other place with in use/need.

Changed to always use need vs in-use. I think need and needed are actually more correct here.

jorgeorpinel

Just a few more precisions but looks good to me otherwise:

jorgeorpinel · 2020-03-18T17:22:32Z

+`--all-commits`, or any combination of them must be provided. Each of them
+corresponds to the current workspace _and_ a set of commits to analyze what
+files, directories and what versions are still needed and should be kept (by
+analyzing DVC-files in those commits).


Suggested change

`--all-commits`, or any combination of them must be provided. Each of them

corresponds to the current workspace _and_ a set of commits to analyze what

files, directories and what versions are still needed and should be kept (by

analyzing DVC-files in those commits).

`--all-commits`, or any combination of them must be provided. Each of them

corresponds to keeping the data for the current workspace and for a

different set of commits (determined by reading the DVC-files in them).

Question: what's the point of this option? To me it sounds like "don't gc anything" except maybe deleted parts of the repo like removed branches, rewritten commits, etc.

DVC cache contains data that is not referenced in any way in Git history. E.g. run do a lot of dvc run, or dvc repro after modifying certain files/settings. If --no-commit is not specified DVC is saving data to cache not matter you do git commit to save DVC-files or not.

We probably mention this.

mentioned a possible use case in the recent version - @jorgeorpinel pls take a look

Makes sense, thanks.

Still this suggestion wasn't applied but I'll address in regular updates.

I put a new paragraph that explains the use case, have you seen it? Or may be I didn't get you specific suggestion ... 🤔

I see it, thanks. May reword a little in regular updates but it's great!

One of the use cases for this option is to safely delete all temporary data DVC cached when dvc run and/or dvc repro were run without committing changes to DVC-files (thus potentially caching data that is not referenced from workspace or Git commits).

Co-Authored-By: Jorge Orpinel <jorgeorpinel@users.noreply.github.com>

…ommits

shcheklein · 2020-03-19T02:50:40Z

@jorgeorpinel @skshetry PTAL again.

skshetry

LGTM 👍

shcheklein · 2020-03-19T03:48:37Z

@jorgeorpinel merging this, feel free to do changes on top of it!

per #1023 (review)

jorgeorpinel · 2020-03-19T07:04:10Z

Great 🎉 Just added a couple more changes in 0f10925 (part of #1072)

per #1023 (comment)

skshetry added the A: docs Area: user documentation (gatsby-theme-iterative) label Feb 25, 2020

skshetry requested review from jorgeorpinel and shcheklein February 25, 2020 15:16

skshetry self-assigned this Feb 25, 2020

shcheklein temporarily deployed to dvc-landing-gc-flag-cha-j6yay1 February 25, 2020 15:16 Inactive

shcheklein reviewed Feb 26, 2020

View reviewed changes

Comment thread public/static/docs/command-reference/gc.md Outdated

shcheklein reviewed Feb 26, 2020

View reviewed changes

Comment thread public/static/docs/command-reference/gc.md

shcheklein reviewed Feb 26, 2020

View reviewed changes

Comment thread public/static/docs/command-reference/gc.md Outdated

shcheklein suggested changes Feb 26, 2020

View reviewed changes

weekly-digest Bot mentioned this pull request Mar 1, 2020

Weekly Digest (23 February, 2020 - 1 March, 2020) #1031

Closed

skshetry requested a review from shcheklein March 2, 2020 08:14

jorgeorpinel reviewed Mar 2, 2020

View reviewed changes

Comment thread public/static/docs/command-reference/gc.md Outdated

This comment has been minimized.

Sign in to view

jorgeorpinel suggested changes Mar 2, 2020

View reviewed changes

jorgeorpinel mentioned this pull request Mar 3, 2020

Reconsider gc implementation treeverse/dvc#2325

Closed

jorgeorpinel suggested changes Mar 3, 2020

View reviewed changes

skshetry mentioned this pull request Mar 9, 2020

gc: make --all-commits flag visible treeverse/dvc#3429

Merged

3 tasks

cmd-ref: document new gc behavior

21572eb

skshetry force-pushed the gc-flag-changes branch from 5611c60 to 21572eb Compare March 16, 2020 05:31

skshetry requested a review from jorgeorpinel March 16, 2020 05:33

shcheklein reviewed Mar 16, 2020

View reviewed changes

Comment thread public/static/docs/command-reference/gc.md Outdated

skshetry and others added 3 commits March 17, 2020 18:27

mention about -w as scope specifier for garbage collection

50156f1

Merge branch 'master' into gc-flag-changes

ab7f625

apply some corrections to the latest gc doc

e530271

shcheklein temporarily deployed to dvc-landing-gc-flag-cha-stdoiq March 18, 2020 05:28 Inactive

skshetry commented Mar 18, 2020

View reviewed changes

jorgeorpinel suggested changes Mar 18, 2020

View reviewed changes

Update content/docs/command-reference/gc.md

e4f1b3c

Co-Authored-By: Jorge Orpinel <jorgeorpinel@users.noreply.github.com>

shcheklein temporarily deployed to dvc-landing-gc-flag-cha-stdoiq March 18, 2020 22:51 Inactive

Update content/docs/command-reference/gc.md

06445b8

Co-Authored-By: Jorge Orpinel <jorgeorpinel@users.noreply.github.com>

shcheklein temporarily deployed to dvc-landing-gc-flag-cha-stdoiq March 18, 2020 22:52 Inactive

Update content/docs/command-reference/gc.md

f90622d

Co-Authored-By: Jorge Orpinel <jorgeorpinel@users.noreply.github.com>

shcheklein temporarily deployed to dvc-landing-gc-flag-cha-stdoiq March 18, 2020 22:53 Inactive

Update content/docs/command-reference/gc.md

e2ea8a7

Co-Authored-By: Jorge Orpinel <jorgeorpinel@users.noreply.github.com>

shcheklein temporarily deployed to dvc-landing-gc-flag-cha-stdoiq March 18, 2020 22:53 Inactive

address reviews feedback, always use need, mention use case for all-c…

49e4f53

…ommits

shcheklein temporarily deployed to dvc-landing-gc-flag-cha-stdoiq March 19, 2020 02:48 Inactive

shcheklein requested a review from jorgeorpinel March 19, 2020 02:49

shcheklein assigned shcheklein and unassigned skshetry Mar 19, 2020

skshetry commented Mar 19, 2020

View reviewed changes

shcheklein merged commit 0381090 into master Mar 19, 2020

skshetry deleted the gc-flag-changes branch March 19, 2020 03:55

jorgeorpinel added a commit that referenced this pull request Mar 19, 2020

cmd ref: imros to gc

0f10925

per #1023 (review)

jorgeorpinel added a commit that referenced this pull request Mar 19, 2020

cmd ref: update gc --all-commits desc

db19c60

per #1023 (comment)

	DVC cache but are no longer needed. With `--cloud` it also removes data in
	DVC cache but are no longer in use. With `--cloud` it also removes data in

Conversation

skshetry commented Feb 25, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

shcheklein left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

This comment has been minimized.

jorgeorpinel left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jorgeorpinel left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shcheklein commented Mar 7, 2020

Uh oh!

skshetry commented Mar 7, 2020

Uh oh!

shcheklein commented Mar 7, 2020

Uh oh!

shcheklein commented Mar 15, 2020

Uh oh!

skshetry commented Mar 16, 2020

Uh oh!

Uh oh!

shcheklein commented Mar 18, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jorgeorpinel left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

shcheklein commented Mar 19, 2020

Uh oh!

skshetry left a comment

Choose a reason for hiding this comment

Uh oh!

shcheklein commented Mar 19, 2020

Uh oh!

jorgeorpinel commented Mar 19, 2020

Uh oh!

Reviewers

Assignees

Labels

jorgeorpinel left a comment •

edited

Loading