pull output is not informative and too massive

[shcheklein]

If I have a few directories it's hard to understand what's going on:

[##############################] 100% Collecting information
[##############################] 100% Analysing status.
[##############################] 100% data/features
Preparing to download data from 'https://remote.dvc.org/get-started'
Preparing to collect status from https://remote.dvc.org/get-started
[##############################] 100% Collecting information
[##############################] 100% Analysing status.
[##############################] 100% data/prepared
Preparing to download data from 'https://remote.dvc.org/get-started'
Preparing to collect status from https://remote.dvc.org/get-started
[##############################] 100% Collecting information
[##############################] 100% Analysing status.

It feels like it's printing the same thing over and over again. It's confusing, I was even though for a moment that this is a bug.

[efiop]

@shcheklein That first part is from where it tries to pull missing dir cache during used cache collection. Need to make it more clear, and, probably optimize it so it doesn't download those files one-by-one during cache collection.

[shcheklein]

@efiop @casperdcl is this one resolved? In the current version I actually see something happening (Multi-threaded messages or something like this blinking), then progress bars that show download process and then nothing. I think we cut everything too aggressively, Multi-threaded is not important enough to emphasize (some message like "Downloading data" should be emphasized). At the end I would keep at least a list of what has been done, may be show a list of files directories with some basic stats - removed/added - similar to what Git is doing.

[efiop]

@shcheklein We've cut some stuff out as a part of initial migration, and a more in-depth reconsideration will be coming later. @casperdcl added assignee to better track this one 🙂

The original issue is caused by two-step pull, where dvc is first pulling .dir cache file to know which files it needs to download later. And that one is still there, but the tqdm migration made it much more bearable. We would want to closer look into this one later.

[efiop]

For the record: When pulling/pushing large number of fairly small files, you see N progress bars flickering in and out of the existence(which is understandable as push/pull takes a short time). Need to come up with a better way of displaying the progress.

[casperdcl]

Think this can be closed as it was fixed by clearing away progress bars upon completion. Actually currently might not be informative enough.
#1838 seems more appropriate. @shcheklein?