Skip to content

fs.callbacks: simplify, ensure None does not break them, lazy richcallbacks#7722

Merged
skshetry merged 1 commit into
treeverse:mainfrom
skshetry:fix-7704
May 10, 2022
Merged

fs.callbacks: simplify, ensure None does not break them, lazy richcallbacks#7722
skshetry merged 1 commit into
treeverse:mainfrom
skshetry:fix-7704

Conversation

@skshetry
Copy link
Copy Markdown
Collaborator

@skshetry skshetry commented May 10, 2022

Fixes #7704.

  1. Simplifies implementation of callbacks, since only call() modifies the progress bars now. Progress bars now use the value/size from fsspec.Callback instead of using their own state.
  2. Ensures that None value in relative_update/absolute_update does not break callbacks. This means that now we can guarantee that value will always be int. This is useful for filesystems that are not very compliant.
  3. Gives the same treatment done in TqdmCallback: make it lazy #7715, makes rich callbacks lazy. Note that we don't use them anywhere yet.
  4. Adds a test for that particular issue.

Note that size can be None, this has to be handled in respective callbacks, TqdmCallback now uses update_to which handles them, RichCallback also handles those well.

@skshetry skshetry added refactoring Factoring and re-factoring ui user interface / interaction A: data-management Related to dvc add/checkout/commit/move/remove labels May 10, 2022
@skshetry skshetry requested a review from a team as a code owner May 10, 2022 12:01
@skshetry skshetry self-assigned this May 10, 2022
@skshetry skshetry requested review from efiop and karajan1001 May 10, 2022 12:01
@skshetry skshetry force-pushed the fix-7704 branch 4 times, most recently from 8ac73b5 to 5ed3d47 Compare May 10, 2022 14:41
@skshetry skshetry merged commit d92c6fe into treeverse:main May 10, 2022
@skshetry skshetry deleted the fix-7704 branch May 10, 2022 14:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A: data-management Related to dvc add/checkout/commit/move/remove refactoring Factoring and re-factoring ui user interface / interaction

Projects

None yet

Development

Successfully merging this pull request may close these issues.

push: dir file not pushed because of progress bar error

1 participant