dvc gc does not remove cache for uncommitted dvc-file

[wmisha]

I removed a dvc-file prior to committing anything, because I accidentally added the wrong file. Afterwards I was not able to remove the cached data.

Steps to reproduce:
dvc add example.txt
rm example.txt.dvc
dvc gc --all-commits (although the other modes of dvc gc are likely affected too)

Expected behavior:
The blob for example.txt is removed from .dvc/cache

Observed behavior:
The blob for example.txt remains in .dvc/cache

Version: 0.88

Platform: Windows 10 x64, exe installation

Notes:

• The unit test for --all-commits doesn't cover a case where a dvc-file is removed before being committed. It only checks that when multiple versions of the dvc-file are present (with some committed into git), only versions that were committed are kept in cache. So it wouldn't have caught this issue.

• Not sure if it's of any significance but in my case, an entry corresponding to my specific file in question remained in the state SQLite database.
  

[efiop]

Hi @hhe !

.dvc/state doesn't matter, it just contains some history.

Are you sure that blob is not used anywhere else in your commit history? Or in other currently present dvc-files?

[wmisha]

I'm sure.

[efiop]

@hhe Not able to reproduce:

#!/bin/bash            
                       
set -e                 
set -x                 
                       
rm -rf myrepo          
mkdir myrepo           
cd myrepo              
git init               
dvc init               
                       
echo foo > foo         
dvc add foo            
rm foo.dvc             
                       
tree .dvc/cache        
                       
dvc gc --all-commits -f
                       
tree .dvc/cache        

produces

+ tree .dvc/cache                                                                              
.dvc/cache                                                                                     
└── d3                                                                                         
    └── b07384d113edec49eaa6238ad5ff00                                                         
                                                                                               
1 directory, 1 file                                                                            
+ dvc gc --all-commits -f                                                                      
WARNING: This will remove all cache except items used in the working tree and all git commits o
f the current repo.                                                                            
+ tree .dvc/cache                                                                              
.dvc/cache                                                                                     
└── d3                                                                                         
                                                                                               
1 directory, 0 files                                                                           

Admittedly, I'm not running it on windows. It would be great, if you could give it a try and see if this script reproduces the issue for you. Might be able to try it on windows myself later.

[efiop]

@hhe Also, you are using a very old dvc version, mind giving it a try with a newer one? E.g. with the pre-release: pip install dvc --pre.

[efiop]

Closing for now. Please let us know if the newer versions work for you.