Import of Test Examples changes matplotlib backend

[uliano]

Expected behavior

Importing example datafiles should not affect matplotlib backend as with 'agg' bad things happen (cfr. [7])

Actual behavior

uliano@dewey:~$ ipython
Python 3.7.1 | packaged by conda-forge | (default, Nov 13 2018, 10:30:07) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.2.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import matplotlib                                                       
In [2]: matplotlib.get_backend()                                                
Out[2]: 'Qt5Agg'
In [3]: from MDAnalysis.tests.datafiles import PSF,DCD                          
In [4]: matplotlib.get_backend()                                                
Out[4]: 'agg'
In [5]: import matplotlib.pyplot as plt                                         
In [6]: plt.plot([1,2])                                                         
Out[6]: [<matplotlib.lines.Line2D at 0xb18bdd860>]
In [7]: plt.show()                                                              
/opt/anaconda3/lib/python3.7/site-packages/matplotlib/figure.py:445: UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.
  % get_backend())

Currently version of MDAnalysis

uliano@dewey:~$ python -c "import MDAnalysis as mda; print(mda.__version__)"
0.19.2
uliano@dewey:~$ python -V
Python 3.7.1
uliano@dewey:~$ sw_vers
ProductName:	Mac OS X
ProductVersion:	10.14.2
BuildVersion:	18C54

[orbeckst]

Fair enough, but honestly I have been struggling to understand how to do this differently because matplotlib does not allow us to change the backend later. How can we ensure that our tests run with the agg backend and not with any of the GUI ones?

[richardjgowers]

We could do a pytest fixture that switches the backend to/from, something like:

@pytest.fixture
def mpl():
    curr = matplotlib.get_backend()
    try:
        matplotlib.use('agg')
        yield
    finally:
        matplotlib.use(curr)

def test_plot(mpl):
    # test with backend temporarily changed

But matplotlib complains about the backend being set after pyplot import, so there's another step I'm missing. Maybe a forced reload of the package?

[jbarnoud]

We could set the backend when we import matplotlib in each test module that needs it, instead of doing the import at the package level in the init. It is not much of a problem if pytest changes the backend, the real issue is that the init changes it which mean we cannot safely import from datafiles.

[uliano]

Now I see where it comes from, as @jbarnoud says; the documentation is full of like examples. For example I stumbled upon this:
https://www.mdanalysis.org/docs/documentation_pages/analysis/contacts.html#examples-for-contact-analysis

[tylerjereddy]

The discussion above seems to suggest the backend adjustment is actually happening for the unit test infrastructure, rather than for the examples in docstrings.

If the main reason for having some kind of top-level data module backend switch to agg is because of docstrings, we might consider using i.e., refguide_check.py instead of vanilla doctests, since the backend switch to Agg happens gracefully / automatically for doctesting that way.

But if it is really unit tests that are the problem refguide doesn't solve the issue. Do we actually CI test our docstrings with vanilla doctest? I'd guess not--standard library doctest is a real pain for scientific code, which is why refguide was created. There's some interest in creating a third-party refguide library so that different packages don't keep vendoring the module.

[tylerjereddy]

There's also switch_backend() if that's needed

[orbeckst]

We should get rid of

mdanalysis/testsuite/MDAnalysisTests/__init__.py

Lines 120 to 128 in 1f4e492

	# Any tests that plot with matplotlib need to run with the simple agg backend
	# because on Travis there is no DISPLAY set. This doesn't warn if we import
	# files from the tests after loading matplotlib. This will remove unnecessary
	# user warnings.
	try:
	import matplotlib
	matplotlib.use('agg', warn=False)
	except ImportError:
	pass

which was included to let the CI run cleanly but presumably we could achieve this in a different manner by, say, setting matplotlibrc on Travis.

[orbeckst]

https://matplotlib.org/3.2.1/tutorials/introductory/usage.html#backends says that we can set

export MPLBACKEND=agg

in our CI environment and then that will overrule anything else. In particular, it allows us to get rid off matplotlib.use().