Skip to content

GDrive improvements and code cleanup#3294

Merged
efiop merged 8 commits into
masterfrom
gdrive-cleanup
Feb 11, 2020
Merged

GDrive improvements and code cleanup#3294
efiop merged 8 commits into
masterfrom
gdrive-cleanup

Conversation

@shcheklein
Copy link
Copy Markdown
Contributor

@shcheklein shcheklein commented Feb 10, 2020

It's a major rewrite now. It was needed since it was very hard an unproductive fixing certain issues w/o addressing some underlying problems with code.

  • Performance of remote id resolving improved. It does not hit API now for "remote path". Affects pull - one API call per file less!
  • Performance of the first run should be improved. We now cache path->id as we create them (case when we push a lot first time we create a remote).
  • Number of retries increased. The previous was not enough with default GDrive limits.
  • Does not create remote path anymore (see the Stop creating a path to the DVC remote root if it does not exist checkbox) in Google Drive support further enhancements #2865
  • Code cleanup.

  • ❗ Have you followed the guidelines in the Contributing to DVC list?

  • 📖 Check this box if this PR does not require documentation updates, or if it does and you have created a separate PR in dvc.org with such updates (or at least opened an issue about it in that repo). Please link below to your PR (or issue) in the dvc.org repo.

  • ❌ Have you checked DeepSource, CodeClimate, and other sanity checks below? We consider their findings recommendatory and don't expect everything to be addressed. Please review them carefully and fix those that actually improve code or fix bugs.

Thank you for the contribution - we'll try to review it as soon as possible. 🙏

@shcheklein shcheklein changed the title [WIP] Gdrive code cleanup [WIP] GDrive improvements and code cleanup Feb 10, 2020
@codecov
Copy link
Copy Markdown

codecov Bot commented Feb 11, 2020

Codecov Report

❗ No coverage uploaded for pull request base (master@2aff0a0). Click here to learn what that means.
The diff coverage is 96.42%.

Impacted file tree graph

@@            Coverage Diff            @@
##             master    #3294   +/-   ##
=========================================
  Coverage          ?   92.83%           
=========================================
  Files             ?      139           
  Lines             ?     8541           
  Branches          ?        0           
=========================================
  Hits              ?     7929           
  Misses            ?      612           
  Partials          ?        0
Impacted Files Coverage Δ
dvc/command/imp.py 86.36% <ø> (ø)
dvc/exceptions.py 97.27% <ø> (ø)
dvc/command/metrics.py 96.82% <ø> (ø)
dvc/command/get.py 91.89% <100%> (ø)
dvc/output/local.py 98.27% <100%> (ø)
dvc/repo/metrics/diff.py 95.38% <100%> (ø)
dvc/repo/diff.py 100% <100%> (ø)
dvc/repo/reproduce.py 98.66% <100%> (ø)
dvc/command/diff.py 92.85% <100%> (ø)
dvc/repo/__init__.py 96.76% <88.88%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2aff0a0...85fef88. Read the comment docs.

@shcheklein
Copy link
Copy Markdown
Contributor Author

@efiop I suggest we review codeclimate and deepsource settings a bit and make the more useful/less noisy for us. E.g. number of arguments per function - if we think that 4 is to aggressive let's increase it a bit?

@shcheklein shcheklein requested review from efiop and maxhora February 11, 2020 03:21
@shcheklein shcheklein changed the title [WIP] GDrive improvements and code cleanup GDrive improvements and code cleanup Feb 11, 2020
Copy link
Copy Markdown
Contributor

@maxhora maxhora left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks great to me! 👍

@efiop efiop merged commit 5e98a70 into master Feb 11, 2020
@delete-merged-branch delete-merged-branch Bot deleted the gdrive-cleanup branch February 11, 2020 17:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants