Skip to content

pypi querry in update_index.py is affected by ratelimit changes  #58

@gnikonorov

Description

@gnikonorov

As per pypi/warehouse#8753, we must now rate limit how we send requests to PyPI

If we don't we receive xmlrpc.client.Fault errors:

$ python3 update_index.py
Traceback (most recent call last):
  File "update_index.py", line 109, in <module>
    main()
  File "update_index.py", line 100, in main
    plugins = sorted(iter_plugins(client, BLACKLIST, consider_classifier=False))
  File "update_index.py", line 41, in iter_plugins
    versions = client.package_releases(name)
  File "/usr/lib/python3.6/xmlrpc/client.py", line 1112, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib/python3.6/xmlrpc/client.py", line 1452, in __request
    verbose=self.__verbose
  File "/usr/lib/python3.6/xmlrpc/client.py", line 1154, in request
    return self.single_request(host, handler, request_body, verbose)
  File "/usr/lib/python3.6/xmlrpc/client.py", line 1170, in single_request
    return self.parse_response(resp)
  File "/usr/lib/python3.6/xmlrpc/client.py", line 1342, in parse_response
    return u.close()
  File "/usr/lib/python3.6/xmlrpc/client.py", line 656, in close
    raise Fault(**self._stack[0])
xmlrpc.client.Fault: <Fault -32500: 'HTTPTooManyRequests: The action could not be performed because there were too many requests by the client. Limit may reset in 1 seconds.'>
$

I am working on fixing this. I should have a PR up today or tomorrow, if that is OK with you @nicoddemus

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions