Python 3.10 compatibility

[elacuesta]

Update some imports and test requirements for py310 compatibility.

Small overlap with #159, I think we can merge this sooner.

[codecov]

Codecov Report

Merging #166 (5adfb9c) into master (34ecb80) will decrease coverage by 0.00%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #166      +/-   ##
==========================================
- Coverage   94.27%   94.27%   -0.01%     
==========================================
  Files          28       28              
  Lines        1958     1957       -1     
==========================================
- Hits         1846     1845       -1     
  Misses        112      112              

Impacted Files	Coverage Δ
scrapinghub/client/frontiers.py	89.25% <ø> (ø)
scrapinghub/client/items.py	100.00% <ø> (ø)
scrapinghub/client/projects.py	94.59% <ø> (ø)
scrapinghub/client/proxy.py	95.71% <ø> (ø)
scrapinghub/client/collections.py	92.30% <100.00%> (ø)
scrapinghub/hubstorage/resourcetype.py	97.39% <100.00%> (-0.02%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 34ecb80...5adfb9c. Read the comment docs.

[elacuesta]

For some reason, any version of pytest greater than 3.7.0 breaks the tests with
requests.exceptions.ConnectionError: HTTPConnectionPool(host='storage.vm.scrapinghub.com', port=80): Max retries exceeded with url: /ids/2222222/spider/hs-test-spider?create=True (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f05c493fbe0>: Failed to establish a new connection: [Errno 113] No route to host')) (see https://github.com/scrapinghub/python-scrapinghub/runs/5484969907)

That is a problem for py310, because of pytest-dev/pytest#8540 (see https://github.com/scrapinghub/python-scrapinghub/runs/5484724225), which is available since 6.2.4

[Gallaecio]

I see E AttributeError: module 'collections' has no attribute 'Iterable' in the latest test failures, which sounds like #160

[elacuesta]

I see E AttributeError: module 'collections' has no attribute 'Iterable' in the latest test failures, which sounds like #160

That particular error should have been fixed by 1f6f39a, I don't see it in the most recent runs.

[Gallaecio]

python-scrapinghub/scrapinghub/client/collections.py

Line 188 in c70137a

not isinstance(keys, collections.Iterable)):

[elacuesta]

🤦
Thanks, should be fixed by a29a80f. The requests.exceptions.ConnectionError problem seems to be still present though 😓

[Gallaecio]

Backporting the Python 3.10 fix I have been able to identify pytest-dev/pytest#3754 as the cause of the issue. Things work with the prior upstream commit.

I have pushed at https://github.com/Gallaecio/pytest/tree/3.7.0-3.7.1-fork the latest working pytest with the Python 3.10 fix backported, in case you want to use that in the meantime.

I will continue looking into it for a more permanent fix.

[elacuesta]

I restored py27 and py35 support, it's not strictly necessary to remove them in order to support py310