dvc remove does not remove neither output files from cache nor the files on remote

[markbudiak]

I have a ML pipeline where I have multiple steps. The outputs are also pushed to the remote storage on aws. I want to delete one step of the pipeline and all its outputs. I used the command:

dvc remove --purge classification.dvc

This removes the dvc file and the outputs from local data directory. Then I ran the command to update dvc also on remote:

dvc push
Surprisingly, the output was still in local cache folder and also on remote.
How can I fix this issue? Did I use the wrong command parameters to update dvc? Or do I need to use another commands?

DVC version: 0.19.12
method of installation: pip
Platform:

• Distributor ID: Ubuntu
• Description: Ubuntu 18.04.1 LTS
• Release: 18.04
• Codename: bionic

[efiop]

Hi @markb21 !

dvc remove is not supposed to remove cache, since it might be used somewhere in the history of your project and you might want to have it. Please take a look at dvc gc command, that collects garbage. E.g. to remove all the cache unused in all tags and branches in your project and on remote use dvc gc -caT.

Thanks,
Ruslan

[efiop]

Also, you are using an old dvc version(current one is 0.19.15), please consider upgrading(i.e. pip install --upgrade dvc).

[markbudiak]

Hi @efiop,
thanks a lot. The command you shared did its job. I was not aware of dvc gc.

[efiop]

Looks like we don't mention it enough in the docs. I'll add a few references ASAP(e.g. to https://dvc.org/doc/commands-reference/remove). Thank you for the feedback!

[markbudiak]

Thanks! Now, I think it makes perfect sense to put a link to a command that can do remote removal.

[efiop]

@markb21 Do you mean add dvc gc -caT to https://dvc.org/doc/commands-reference/gc as an example, right?

[markbudiak]

@efiop Sorry for very late reply but that would help a lot to make it more straightforward.

[efiop]

@markb21 No worries :) Created treeverse/dvc.org#154 to track progress on it. Thanks!