MRG, MNT: Migrate to nbclient for notebook testing

[GuillaumeFavelier]

Checks that the CIs are green on Azure for the notebook 3d backend.

[GuillaumeFavelier]

Now there is a warning on Azure for notebook testing:

FutureWarning: Method cleanup(connection_file=True) is deprecated, use cleanup_resources(restart=False).

[GuillaumeFavelier]

This is probably because of the new version of jupyter_client:

[GuillaumeFavelier]

Bumping nbconvert solved the warning but now pytest-notebook is not ready for the change:

AttributeError: 'ExecuteCoveragePreprocessor' object has no attribute 'setup_preprocessor'

[GuillaumeFavelier]

I think it's fine since nbconvert 6.0.0 is not officially released yet. We could probably ignore for now and report the change on pytest-notebook ?

[GuillaumeFavelier]

This is trickier than it seems. There is an API change in nbconvert 6.0.0 leading to some part of the code to be split in nbclient.

[GuillaumeFavelier]

As we do not gather coverage informations from this notebook testing, I could consider migrating to nbclient instead.

[GuillaumeFavelier]

I reported the FutureWarning in jupyter/nbclient#89.

For now, we can just pin jupyter_client<6.1.5.

[GuillaumeFavelier]

What do you think @agramfort, @larsoner ?

[agramfort]

I am confused. Why removing pytest-notebook?

[GuillaumeFavelier]

We don't use the coverage information and it's not ready yet for nbconvert 6.0.0.

[larsoner]

What do you mean we don't use it? It should be reported like all other runs (other than style)

[GuillaumeFavelier]

It should be reported like all other runs

I was not aware of that

[agramfort]

coverage report or execution? I can live without coverage but the notebook definitely needs to be executed by pytest

…

[GuillaumeFavelier]

nbclient executes the notebook test.ipynb in the test_notebook_3d_backend() function of the test_notebook.py file.

I'm now completely lost for coverage.

[agramfort]

if test.ipynb runs on CIs than I am happy

…

[larsoner]

nbclient executes the notebook test.ipynb in the test_notebook_3d_backend() function of the test_notebook.py file.

I'm now completely lost for coverage.

As long as it does this in the same Python process, I think coverage should work. And based on the coverage numbers for the notebook backend, it should be working.

https://codecov.io/gh/mne-tools/mne-python/src/master/mne/viz/backends/_notebook.py

[larsoner]

Closing in favor of #7943