Failure to pull after importing a git repo

[charlesbaynham]

Please provide information about your setup
DVC version(i.e. dvc --version), Platform and method of installation (pip, homebrew, pkg Mac, exe (Windows), DEB(Linux), RPM(Linux))

DVC 0.86.4, conda installation, Windows.

See https://discuss.dvc.org/t/importing-from-a-git-repo-then-pulling/320 for background.

After importing a directory from a git repository (not a dvc repository), dvc pull fails with ERROR: unexpected error - 'ExternalGitRepo' object has no attribute 'cache'.

[charlesbaynham]

Script to reproduce:

mkdir git_repo
cd git_repo
git init
mkdir a_dir
touch a_dir/test.txt
git add .
git commit -m "Init git repo"

cd -
mkdir dvc_repo
cd dvc_repo
git init
dvc init
git add .
git commit -m "Init"
dvc import ../git_repo a_dir
dvc pull

[efiop]

Thanks for the bug report @charlesbaynham ! We'll take a look ASAP.

[skshetry]

@jorgeorpinel, @efiop, how should this behave for git repo? Not pull at all or like dvc update? Could not find anything on https://man.dvc.org/pull about this.

[charlesbaynham]

Am I right in thinking that dvc repos don't put imported files into their cache / remote? So a pull will always need to access the original source?

[jorgeorpinel]

@charlesbaynham it actually does put it in the cache. Continuing your script:

$ cat a_dir.dvc
...
  repo:
    url: ../git_repo
    rev_lock: 314393c2a7b82004302d5b7f219e724522db88ca
outs:
- md5: 59446444eae8d3c063f64fefd37b43b5
...
$ tree .dvc/cache 
.dvc/cache
├── 59
│   └── 446444eae8d3c063f64fefd37b43b5.dir
└── d4
    └── 1d8cd98f00b204e9800998ecf8427e
$ cat .dvc/cache/59/446444eae8d3c063f64fefd37b43b5.dir 
[{"md5": "d41d8cd98f00b204e9800998ecf8427e", "relpath": "test.txt"}]

But shouldn't push it to the remote storage, indeed.

@skshetry in this case, I don't understand why dvc pull is even trying to pull from this import stage, since the output is already in cache ^ (and workspace).

https://dvc.org/doc/command-reference/pull

I think that's the current p0 bug.

• Whether there are other problems with Git-tracked imported files, we would need to extensively Q/A re-importing and updating them... And probably dvc fetch\pull and dvc push as well, just in case.

[efiop]

@charlesbaynham it actually does put it in the cache. Continuing your script:

It actually doesn't, just that .dir file, which is more of a temporary file than a proper cache in that particular case. So we will indeed need to have access to the original source.

[efiop]

@skshetry It should git clone (as we usually do) and just copy stuff over, same as if you would've re-imported it again.

[jorgeorpinel]

It actually doesn't

OK. But in my script you can see that both .dvc/cache/59/446444e...b5.dir (a_dir) AND .dvc/cache/d4/1d8cd98... (a_dir/test.txt) are in cache... what am I missing? 🙂