Test failures due to wrong scipy version being installed

[zemanj]

Currently, all NUMPY_VERSION=1.10.4 tests fail on Travis.

The problem is that conda installs a too old scipy version:

scipy: 0.19.1-py35_blas_openblas_202 conda-forge [blas_openblas]

even though we require scipy>=1.0.
Later, when $BUILD_CMD is evaluated, the build notices the missing requirement

Collecting scipy>=1.0.0 (from MDAnalysis==0.19.3.dev0)

And a little further below:

ERROR: scipy 1.3.0 has requirement numpy>=1.13.3, but you'll have numpy 1.10.4 which is incompatible.

As expected, the tests then fail due to numpy/scipy incompatibilities.

[zemanj]

I just found out that conda used to install wrong package versions for quite a while (see here), which never caused problems because the pip version being installed by the MDA build (see here and here) did not have incompatibilities. Now, with scipy 1.3, this no longer seems to work.
The reason seems to be where version numbers are stripped from CONDA_DEPENDENCIES in ci-helpers. Looks like the pin file created beforehand is not respected.

[orbeckst]

Can we just bump the numpy requirement to be compatible with scipy >= 1.0?

@tylerjereddy do you have some insights in how necessary it is to maintain numpy 1.10.4 compatibility?

[orbeckst]

@zemanj – if we required numpy >= 1.13.3 would everthing be ok?

If this is the case and no-one else is chiming in (@MDAnalysis/coredevs , anyone?) I would say we conserve developer time and just bump the requirement instead of debugging ci-helpers/conda version hell.

[orbeckst]

Btw, does this problem originate from our setup.py requring scipy >= 1.0

mdanalysis/package/setup.py

Line 592 in f83498e

'scipy (>=1.0.0)', 'matplotlib (>=1.5.1)'],

but on travis we don't specify these dependencies explicitly:

mdanalysis/.travis.yml

Line 30 in f83498e

- CONDA_MIN_DEPENDENCIES="mmtf-python mock six biopython networkx cython joblib matplotlib scipy griddataformats hypothesis gsd codecov"

?

Should we just be consistent between the two files? Maybe that fixes things or at least fails earlier?

[zemanj]

I have asked this question already before in another issue but either nobody knew the answer or didn't bother. I'd really like to know on which basis the minimum numpy/python versions we support are determined. I personally would love to see the minimum numpy version bumped to 1.13.3.

…

On June 18, 2019 10:48:36 PM GMT+02:00, Oliver Beckstein ***@***.***> wrote: @zemanj – if we required numpy >= 1.13.3 would everthing be ok?> > If this is the case and no-one else is chiming in ***@***.***/coredevs , anyone?) I would say we conserve developer time and just bump the requirement instead of debugging ci-helpers/conda version hell.> > -- > You are receiving this because you were mentioned.> Reply to this email directly or view it on GitHub:> #2272 (comment)

[zemanj]

@orbeckst Yeah we should specify the version numbers explicitly and see if that helps. I'm for bumping rhe minimum numpy version anyway unless somone argues against it.

…

On June 18, 2019 10:52:25 PM GMT+02:00, Oliver Beckstein ***@***.***> wrote: Btw, does this problem originate from our `setup.py` requring scipy >= 1.0 https://github.com/MDAnalysis/mdanalysis/blob/f83498ec6b5cc546a0d75598a4d1d6ede6fe2782/package/setup.py#L592 but on travis we don't specify these dependencies explicitly: https://github.com/MDAnalysis/mdanalysis/blob/f83498ec6b5cc546a0d75598a4d1d6ede6fe2782/.travis.yml#L30 ? Should we just be consistent between the two files? Maybe that fixes things or at least fails earlier? -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #2272 (comment)

[orbeckst]

Just sent an email to the dev list... give it a day and if no-one objects, let's go ahead and bump it. We can't have all our PRs stalled because of that.

[orbeckst]

No-one is against bumping the numpy minimal required version. @zemanj do you want to do the honors (I mean, a quick PR with the appropriate CHANGELOG entry)?

[zemanj]

@orbeckst I'd love to, but I'm on vacation right now and only have my phone with me. I won't be available until Sunday next week. If nobody else has jumped in by then, I'll do it.

…

On June 21, 2019 10:29:56 AM GMT+02:00, Oliver Beckstein ***@***.***> wrote: No-one is against bumping the numpy minimal required version. @zemanj do you want to do the honors (I mean, a quick PR with the appropriate CHANGELOG entry)? -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #2272 (comment)