pypy 3.8 fails to import cloudpickle

[dand-oss]

pypy 3.8 release 7.3.6 does not find c extension _pickle

# python
Python 3.8.12 (279d80ac2079, Oct 17 2021, 05:25:30)
[PyPy 7.3.6 with GCC 7.3.1 20180303 (Red Hat 7.3.1-5)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>> import cloudpickle
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/pypy/lib/pypy3.8/site-packages/cloudpickle/__init__.py", line 4, in <module>
    from cloudpickle.cloudpickle import *  # noqa
  File "/pypy/lib/pypy3.8/site-packages/cloudpickle/cloudpickle.py", line 57, in <module>
    from .compat import pickle
  File "/pypy/lib/pypy3.8/site-packages/cloudpickle/compat.py", line 13, in <module>
    from _pickle import Pickler  # noqa: F401
ModuleNotFoundError: No module named '_pickle'
>>>> 

[p-fernandes]

I would advise to drop the import of _pickle now.

[bartbroere]

Thanks for fixing this! Any idea when it might be included in a release?

(And for anyone googling it, symlinking pickle.py to _pickle.py will do the trick temporarily:

ln -s /opt/pypy/lib/pypy3.8/pickle.py /opt/pypy/lib/pypy3.8/_pickle.py

)

[schettino72]

@bartbroere really? just symlinking fixed the problem?

That did not work for me.

[bartbroere]

@bartbroere really? just symlinking fixed the problem?

That did not work for me.

Maybe I should have tested it a bit more, but we hit this part of cloudpickle through scikit-learn and joblib. When importing a metric from sklearn.metrics PyPy failed on the missing _pickle module. Since our code didn't actually use joblib or cloudpickle, the main thing it fixed was the errors on import of our metric.

Because joblib includes a copy of cloudpickle just pip installing the master here wasn't an option. For your use case, perhaps pip installing the current master is a better idea, since it's already fixed, just not yet released.

[lesteve]

I think this can be closed, since it has been fixed by #469.

[mattip]

Also #480 added pypy3.9 to the CI for testing, which is passing.

[jakirkham]

Thanks all! 🙏

Closing as resolved