Rate limit requests to PyPI due to recent PyPI changes#59
Rate limit requests to PyPI due to recent PyPI changes#59nicoddemus merged 4 commits intopytest-dev:masterfrom
Conversation
nicoddemus
left a comment
There was a problem hiding this comment.
Thanks a lot @gnikonorov!
Left a few comments, let me know what you think!
|
|
||
| BLACKLIST = {"pytest-nbsmoke"} | ||
|
|
||
| class RateLimitedServerProxy: |
There was a problem hiding this comment.
Perhaps we can use some tenacity decorators around the actual proxy instead of implementing a custom logic?
@retry(wait=wait_fixed(30), stop=stop_after_attempt(10))
def browse(self, classifiers):
return self._server_proxy.browse(classifiers)I understand the logic is not as clever as what you implemented, which looks at the actual message to decide what to do, but this might be good enough.
But of course if the plan is to throw this away in favor of a reimplementation using PyPIJSON, I'm happy to have this in the mean time.
There was a problem hiding this comment.
The plan would be to eventually deprecate this in favor of PyPIJSON
Thank you for sharing tenacity! I didn't know this existed. I think we should look into using it for the proper migration
|
@nicoddemus - just updated the PR with your feedback and tested locally. I think this is good to go, if you're ok with it |
nicoddemus
left a comment
There was a problem hiding this comment.
LGTM, thanks again @gnikonorov!
Rate limit how we send requests to PyPI in
update_index.pydue to pypi/warehouse#8753.We should ideally move to https://wiki.python.org/moin/PyPIJSON, but this is a suitable intermediary. After this merges, I will open an issue to make the migration
I tested everything locally, and was able to successfully update my index.json with these code changes
Closes #58