Skip to content

progress when collecting directories with giant number of files #2770

@efiop

Description

@efiop

Was adding one particularly large dataset of a few hundred G with lots of files and saw dvc appear to be frozen when it was collecting all the files to start adding them around https://github.com/iterative/dvc/blob/0.67.0/dvc/remote/base.py#L196 . Would be nice to either speed it up so it is not noticable(which I really doubt is possible with that amount of files to walk through) or at least provide a progress bar to keep the user updated.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementEnhances DVCp2-mediumMedium priority, should be done, but less importantuiuser interface / interaction

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions