import Pickler class directly from the pickle module

[jvesely]

cloudpickle/cloudpickle/compat.py

Line 13 in 7ddb089

from _pickle import Pickler # noqa: F401

uses _pickle module to import Pickler. However, as per python 3.8 documentation[0] this class is available directly in the pickle module.

from pickle import Pickler

works ok and it is equivalent to _pickle.Pickler

$ python 
Python 3.9.7 (default, Nov  1 2021, 14:08:06) 
[GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from _pickle import Pickler as p
>>> import pickle
>>> p is pickle.Pickler
True
>>>

[0] https://docs.python.org/3.8/library/pickle.html#pickle.Pickler

[jvesely]

This also fixes issues with pypy3.8:

    import sys
    
    
    if sys.version_info < (3, 8):
        try:
            import pickle5 as pickle  # noqa: F401
            from pickle5 import Pickler  # noqa: F401
        except ImportError:
            import pickle  # noqa: F401
            from pickle import _Pickler as Pickler  # noqa: F401
    else:
        import pickle  # noqa: F401
>       from _pickle import _Pickler as Pickler  # noqa: F401
E       ModuleNotFoundError: No module named '_pickle'

../pypy38-venv/lib/pypy3.8/site-packages/joblib/externals/cloudpickle/compat.py:13: ModuleNotFoundError

[pierreglaser]

Hi @jvesely, Is there a rationale for changing from _pickle import Pickler to from pickle import Pickler, other than fixing cloudpickle on pypy 3.8? If not, i'd rather keep this line, which need not be changed for other implementations than pypy, and branch the behavior for pypy in other to restore compatibility between cloudpickle, and pypy >= 3.8

[jvesely]

Hi @jvesely, Is there a rationale for changing from _pickle import Pickler to from pickle import Pickler, other than fixing cloudpickle on pypy 3.8? If not, i'd rather keep this line, which need not be changed for other implementations than pypy, and branch the behavior for pypy in other to restore compatibility between cloudpickle, and pypy >= 3.8

This is not specific to version >= 3.8. Python 3.7 includes pickler.Pickler [0] and the same short script proves that _pickle.Pickler and pickle.Pickler. The same is true for Python3.6 as well.[1]

according to the python3 specs [2]: "Users should always import the standard version,". Using _pickle is just a non-portable way of doing the same thing.

This non-portability goes beyond PyPY. The same docs [2]: "In Python 3.0, the accelerated versions are considered implementation details of the pure Python versions." so the names and availability of any _pickle attributes can change between CPython versions.

[0] https://docs.python.org/3.7/library/pickle.html#pickle.Pickler
[1] https://docs.python.org/3.6/library/pickle.html#pickle.Pickler
[2] https://docs.python.org/3.1/whatsnew/3.0.html#library-changes

[bsnresearcher]

I hope that this problem will be solve.
Scikit-learn package uses cloudpickle inside. And it causes the same above problem.

Perhaps, there are a lot of people want to use pypy because their code like using scikit-learn that scientific calculation code may require a lot of calculation so need speed up.

So solved this will contribute to their work a lot.

[jvesely]

@pierreglaser any ETA on a fix for this? It's rather straightforward. I can open a PR if needed.

[pierreglaser]

Hey @jvesely, PR welcome, I'll review it.