JOSS Review [crvernon]: Fail on import after pip install from PyPI

[crvernon]

Review reference: openjournals/joss-reviews#3294

Environment

Mac OSX
Tested with clean pyenv virtual environment for Python versions 3.9.4 and 3.6.8 with same result

Description

During install using pip install atlite, I notice that dask is installing v2.30.0:

Collecting dask<20.04.0,>=0.18.0
  Downloading dask-2.30.0-py3-none-any.whl (848 kB)
     |████████████████████████████████| 848 kB 2.9 MB/s

which is not in the bounds of your target version dask<20.04.0,>=0.18.0

Therefore, when importing after install:

import atlite

the error is raised:

Python 3.9.4 (default, May 12 2021, 10:46:22)
[Clang 10.0.0 (clang-1000.10.44.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import atlite
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/d3y010/repos/joss/atlite/atlite/__init__.py", line 16, in <module>
    from .cutout import Cutout
  File "/Users/d3y010/repos/joss/atlite/atlite/cutout.py", line 35, in <module>
    from .data import cutout_prepare, available_features
  File "/Users/d3y010/repos/joss/atlite/atlite/data.py", line 18, in <module>
    from dask import delayed, compute
  File "/Users/d3y010/.pyenv/versions/py3.9.4_atlite/lib/python3.9/site-packages/dask/delayed.py", line 7, in <module>
    from tlz import curry, concat, unique, merge
ModuleNotFoundError: No module named 'tlz'

Also, your readthedocs page here: https://atlite.readthedocs.io/en/latest/installation.html#required-packages specifies dask unbounded for higher versions at (0.18.0+)

I'll dig into this a bit more tomorrow. May be easy to clear up because your specification looks correct to me.

[FabianHofmann]

Hey @crvernon, thanks starting the review!
Indeed, I can reproduced the error. The version requirement for dask in the setup.py is wrong, it should be <21.04. Then the toolz package is in the dask requirements. I have to update the setup.py and make a new release with fixed requirements.

Also the conda recipe neeeds updated requirements due to the new xarray version for which the atlite tests break.

Sorry for the inconvenience but these packages are often updated and one has catch up with them properly.

[crvernon]

@FabianHofmann no problem. Just let me know when the changes have been made and I'll try the install again. By the way, dask has also now moved to a different versioning semantic that uses calendar versioning for newer releases (mentioned in the change log here: https://docs.dask.org/en/latest/changelog.html#id9). This may be problematic for setting version ranges with dask in the future.

[crvernon]

@FabianHofmann I was able to fix the issue by specifying the dask install as dask[complete] without the need to tag a version. So in the install_requires section of your setup.py:

    install_requires=['numpy',
                      'scipy',
                      'pandas>=0.22',
                      'bottleneck',
                      'numexpr',
                      'xarray>=0.16',
                      'netcdf4',
                      'dask[complete]',
                      'requests',
                      'rtree',
                      "pyyaml",
                      'rasterio>=1.0',
                      'shapely',
                      'progressbar2',
                      'pyproj>=2',
                      'geopandas',
                      'cdsapi'],

Then I am able to import without issue.

[crvernon]

@euronion Just following up to see if there has been any resolve on this issue. Thanks!

[euronion]

@crvernon Not yet finished, it's an active WIP. We'll keep you posted.

[euronion]

Should now be fixed via #144 .