[WIP] Use conda-forge packages when creating environment

[hoechenberger]

Use conda-forge channel, which has up-to-date versions of most of our dependencies, so we won't need to install them via pip. This should reduce the risk of ending up with an inconsistent environment, esp. if users install packages after initial environment creation.

x-ref #7945

cc @cbrnr

[hoechenberger]

I believe we need to create a pyvistaqt feedstock on conda-forge to avoid pulling in another PyQt5 dependency via pip.

[cbrnr]

The fact that conda and pip packages for PyQt5 are named differently has caused me so much headache. I have no idea why conda insists on naming it pyqt.

[hoechenberger]

The fact that conda and pip packages for PyQt5 are named differently has caused me so much headache. I have no idea why conda insists on naming it pyqt.

Yeah I know. But there are workarounds. Also I was thinking about just creating a meta package named PyQt5… but I doubt it would be accepted in conda-forge.

[cbrnr]

Which workaround works with setup.py and requirements.txt (i.e. pip)? I solved this problem by only requiring QtPy, which fortunately has the same package name in conda and PyPI. However, this is not ideal, because users have to manually install PyQt5 or PySide2 - but maybe that's a good thing and this solution isn't that bad after all.

[cbrnr]

Regarding pyvistaqt, you can still install via pip install pyvistaqt --no-deps to avoid pulling in another PyQt5 copy.

[hoechenberger]

Which workaround works with setup.py and requirements.txt (i.e. pip)?

No idea, I never use requirements.txt; but in PsychoPy's setup.py we worked around the conda <> PyPI naming conflicts like this:
https://github.com/psychopy/psychopy/blob/6e6e62f6bcd3a6d359664ea8020fdeefd7d73ac6/setup.py#L25-L42

Regarding pyvistaqt, you can still install via pip install pyvistaqt --no-deps to avoid pulling in another PyQt5 copy.

I don't think this works in a conda environment file, unfortunately.

[cbrnr]

I don't think this works in a conda environment file, unfortunately.

Yes, seems like this is not that easy because it uses a pip requirements file in the background, which also doesn't support adding flags: conda/conda#6805

[hoechenberger]

I believe we need to create a pyvistaqt feedstock on conda-forge

I've started working on a recipe:
conda-forge/staged-recipes#12040

[hoechenberger]

I believe the problems with doc building and on Windows will be resolved by #7943

[drammock]

is there a reason to still use numpydoc master now that they have a 1.0 release?

[hoechenberger]

I have no idea. @larsoner?

[hoechenberger]

Rebased on master as #7943 has been merged.

[GuillaumeFavelier]

Apparently vtk 9.0.1 is not live in conda-forge?

ResolvePackageNotFound: 
  - vtk[version='>=9.0.1']

[hoechenberger]

Apparently vtk 9.0.1 is not live in conda-forge?

Yes, the build didn't succeed:
conda-forge/vtk-feedstock#137

[hoechenberger]

Until conda-forge/mayavi-feedstock#43 has been merged, pysurfer -- which depends on mayavi -- cannot be installed into the environment (which is supposed to deploy VTK 9.0.1)

[hoechenberger]

Apparently vtk 9.0.1 is not live in conda-forge?

it is now :)

[GuillaumeFavelier]

May I suggest to use (at least) this patch to enable GL rendering in the azure job?

Patch

diff --git a/azure-pipelines.yml b/azure-pipelines.yml
index 02fc0c718..bd01c0a30 100644
--- a/azure-pipelines.yml
+++ b/azure-pipelines.yml
@@ -277,8 +277,15 @@ jobs:
   pool:
     vmImage: 'ubuntu-18.04'
   variables:
+    DISPLAY: ':99'
     CONDA_ENV: 'environment_conda-forge.yml'
   steps:
+  - bash: |
+      sudo apt install libxkbcommon-x11-0 xvfb tcsh libxcb*
+    displayName: 'Install Ubuntu dependencies'
+  - bash: |
+      /sbin/start-stop-daemon --start --quiet --pidfile /tmp/custom_xvfb_99.pid --make-pidfile --background --exec /usr/bin/Xvfb -- :99 -screen 0 1400x900x24 -ac +extension GLX +render -noreset;
+    displayName: 'Spin up Xvfb'
   - bash: |
       set -e
       wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh -O miniforge.sh --progress=dot:mega

[GuillaumeFavelier]

traits is still needed somehow even though mayavi is not in the environment:

    @contextmanager
    def traits_test_context():
        """Context to raise errors in trait handlers."""
>       from traits.api import push_exception_handler
E       ModuleNotFoundError: No module named 'traits'

[hoechenberger]

Thanks @GuillaumeFavelier! Let's see…

[larsoner]

Do we still need/want this now that #7971 is the plan going forward?

[drammock]

Do we still need/want this now that #7971 is the plan going forward?

I think there's still an open question about whether to offer an advanced install option for devs/contributors that has its own environment file. Potentially that recipe could include the work here.