Feature/qt_plot_2

[MerlinSmiles]

New PR from my fork so that we can continue to work on PR #290.

@peendebak @giulioungaretti

#######

Fixes the Qt part of #266
Fixes #259 I think
Fixes the Qt part of #217
Fixes #34 since setGeometry is now directly available and _repr_png_ exists
Fixes #289 I implemented it here too...

Changes proposed in this pull request:

• Use pyqtgraph Docks for subplots
• Expose some of the Dock-functionality to QtPlot.add()
• Add transpose argument to QtPlot.add() (flip x, y axes, and transpose z)
• Fix plotting when x, y, z datasets are defined
• Remove Javascript update widget which is only used for update timing, replaced with native Qt timer
• Add PNG representation of the plot windows
• Add plotting tutorial
• Add get default array title currently the location and array_id, maybe it should be reduced to the name and array_id, in the case where name exists. however, location is nice to later relate screenshots to the right dataset.
• Add get_units
• renamped update to update_data in base.py so it wont interfere with update() from the Qwidget
• removed pyqtgraph remote plotting stuff

I'm leaving for a holiday now, but it would be great if someone could check this over, and give some input

This PR is mostly meant to fix some of the urgent issues I came across, there is still a lot to do but for that we need to refactor the whole plotting system, which seems somewhat complicated right now. Make it more modular and allow for easy additions...

[AdriaanRol]

@MerlinSmiles , I recently updated our code to use the qcodes plotting for everything. However we do not use datasets but rather add x and y arrays manually (and providing the right labels as well).

I was wondering if you preserve this backwards capability, additionally, I think it is a good idea if the qcodes plot monitor supports simple x-y plotting for modularity and simplicity reasons.
My guess is that you do and I only need minor updates to my code to check this out but but I thought I'd check before I miss some detail.

[MerlinSmiles]

@AdriaanRol I dont think I have touched any of that...

[AdriaanRol]

Cool, I'll try to check this out this weekend. Seems this PR is in need of some merge support :)

[AdriaanRol]

Hi Merlin,

I couldn't resist trying it out but I ran into a little problem trying the most trivial example.

The following code snippet works on perfectly on master

from qcodes.plots.pyqtgraph import QtPlot
p1 = QtPlot()
x = np.linspace(0,10,100)
y = np.sin(x)
p1.add(x=x, y=y)

However when I check out your PR, I get the error I pasted below. I think this should be pretty easy to solve (adding a default empty string as title should do).

Edit: I tried running your example notebook and I ran into the exact same problem, which I find kind of strange (exact same error message). I am a bit puzzled because if I check out the master branch everything works fine for me and seeing that the notebook I pull in has logged output I can only conclude that it works for you. However, the type of bug I get suggest a simple coding bug due to a changed API and not a weird graphics install type bug.

In any case I'd be interested to figure out if you can run this version (exactly as it is in the PR).

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-15-6f8091d6c7e8> in <module>()
      3 x = np.linspace(0,10,100)
      4 y = np.sin(x)
----> 5 p1.add(x=x, y=y)

/Users/Adriaan/GitHubRepos/DiCarloLab_Repositories/Qcodes/qcodes/plots/base.py in add(self, updater, *args, **kwargs)
     88         self.expand_trace(args, kwargs)
     89         self.add_updater(updater, kwargs)
---> 90         return self.add_to_plot(**kwargs)
     91 
     92     def add_to_plot(self, **kwargs):

/Users/Adriaan/GitHubRepos/DiCarloLab_Repositories/Qcodes/qcodes/plots/pyqtgraph.py in add_to_plot(self, subplot, **kwargs)
    294                 title = self.get_default_array_title(kwargs['x'])
    295 
--> 296         subplot_object.setTitle('#{} - {}'.format(subplot, title or 'Plot'))
    297 
    298         if kwargs.get('clear', False):

AttributeError: 'Dock' object has no attribute 'setTitle'

[peendebak]

@AdriaanRol Which version of pyqtgraph are you using? You need the latest one for this pr.

[giulioungaretti]

@AdriaanRol @peendebak not only for this PR but if you follow master you need the last version as well!

[AdriaanRol]

@giulioungaretti , I made sure my fork of the master was up to date :).
@MerlinSmiles I'll check the pyqtgraph version later.

[AdriaanRol]

@peendebak , @MerlinSmiles I'm running pyqtgraph 0.9.10. This is the same as listed in the main qcodes install page and as far as I can tell also the latest (http://www.pyqtgraph.org/).

Let me know if I should be using any other version (or what you use to run this notebook).

[MerlinSmiles]

@AdriaanRol maybe you want to pip upgrade your pyqtgraph to 0.10 https://pypi.python.org/pypi/pyqtgraph ?

[peendebak]

@AdriaanRol Yes, you should:

https://github.com/QCoDeS/Qcodes/blob/master/setup.py#L29

[giulioungaretti]

@MerlinSmiles @eendebakpt @AdriaanRol ah, but ofc if one checks out this brach/fork the setup.py uses the old versoin still :( or actually a version that does not exist :(

[AdriaanRol]

Ok, I've updated pyqtgraph to 0.10.0. I still run into loads of bugs when I run the example notebook.
As I can proceed by commenting out certain commands I'll try to give a short summary.

• In the first cell I try to run qc.show_subprocess_widget(). This raises an error AttributeError: module 'qcodes' has no attribute 'show_subprocess_widget'. In the past this used to work, I assume this is just deprecated in the current master?

• Command cell 5 raises another error on the last line:

# The plot-window has a PNG representation, thus the folowing line grabs a "screenshot" of the window:
plotQ

Traceback attached at bottom of post.
Commenting this out I can continue.

• I can add the ampltidue to the plot in cell 6 (plots correctly). However trying to use the PNG representation of cell 7 raises the same error as before (see below). In fact whenever I return the object I get this "grab" error. Note that this does not occur when I go back to the latest version of the master.

• Separately from the example notebook, if I create an empty window and give it a title p1 = QtPlot(windowTitle='MyTitle'), I get an error.

• Refering to the pyqtgraph version, 0.10.0 is indeed listed in the dependencies, however the main qcodes page (readme.rst) still indicates 0.9.xx.

Summary

I find a lot of bugs in stuff that used to work fine and I cannot easily get new things to work on a configuration that used to run everything (my macbook).

I would really like to use the improved plotting and see all the new features. I suggest the following changes.

• Add some tests to the plotting. We may not be able to check if it plots correctly but at least we can check if no errors are raised in functionality that should work.
• Make sure the basic functionality remains working (I am quite happy with the current qcodes plotting).

I'd also be interested to see if people can run the current version in the PR, I'd be curious to understand why this is so hard (altough # graphics.... )

Appendix tracebacks

Error when return the plot object in the notebook.

AttributeError                            Traceback (most recent call last)
/usr/local/lib/python3.5/site-packages/IPython/core/formatters.py in __call__(self, obj)
   341             method = _safe_get_formatter_method(obj, self.print_method)
   342             if method is not None:
--> 343                 return method()
   344             return None
   345         else:

/Users/Adriaan/GitHubRepos/DiCarloLab_Repositories/Qcodes/qcodes/plots/pyqtgraph.py in _repr_png_(self)
   689         QtWidgets.QApplication.processEvents()
   690 
--> 691         image = self.grab(self.area.contentsRect())
   692 
   693         byte_array = QByteArray()

AttributeError: 'QtPlot' object has no attribute 'grab'

Error on creating a new window with a title :

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-5-e35396d20705> in <module>()
      1 from qcodes.plots.pyqtgraph import QtPlot
----> 2 p1 = QtPlot(windowTitle='None')
      3 x = np.linspace(0,10,100)
      4 y = np.sin(x)
      5 p1.add(x=x, y=y)

/Users/Adriaan/GitHubRepos/DiCarloLab_Repositories/Qcodes/qcodes/plots/pyqtgraph.py in __init__(self, figsize, figposition, interval, window_title, theme, show_window, parent, *args, **kwargs)
    159 
    160         if args or kwargs:
--> 161             self.add(*args, **kwargs)
    162 
    163         QtWidgets.QApplication.processEvents()

/Users/Adriaan/GitHubRepos/DiCarloLab_Repositories/Qcodes/qcodes/plots/base.py in add(self, updater, *args, **kwargs)
     86         """
     87         # TODO(giulioungaretti): replace with an explicit version, see expand trace
---> 88         self.expand_trace(args, kwargs)
     89         self.add_updater(updater, kwargs)
     90         return self.add_to_plot(**kwargs)

/Users/Adriaan/GitHubRepos/DiCarloLab_Repositories/Qcodes/qcodes/plots/base.py in expand_trace(self, args, kwargs)
    266         # reset axletters, we may or may not have found them above
    267         axletters = 'xyz' if 'z' in kwargs else 'xy'
--> 268         main_data = kwargs[axletters[-1]]
    269         if hasattr(main_data, 'set_arrays'):
    270             num_axes = len(axletters) - 1

KeyError: 'y'

[peendebak]

@AdriaanRol We use this PR in our day to day work, so this issues you find are probably specific to your setup or to the notebook.

• show_subprocess_widget is not loaded, depending on your config settings. Loaded that function causes issues on non-notebook only load show_subprocess_widget when in notebook #366
• The argument windowTitle was renamed to window_title. The error message you get is a bit cryptic, because the function interprets the windowTitle argument as one of the variables to plot. Maybe we can change the function so that we get an error like windowTitle, unknown argument

[MerlinSmiles]

@AdriaanRol
adding to @peendebak, for the plotting error

--> 691         image = self.grab(self.area.contentsRect())
   692 
   693         byte_array = QByteArray()

AttributeError: 'QtPlot' object has no attribute 'grab'

Do you have PyQt5 installed? since the switch to qtpy (that allows using pyqt4, pyqt5, or pyside(?)) i guess we have a hard time adding pyqt5 or pyside as dependency? No idea how that works (@Unga)
However, since pyqtgraph 0.10 supports pyqt5 we can finally use the latest qt, and I think we should.

For the interpretation of a non existing argument as an dataset, that is how the original was desined, you probably get the same error in the old qtplot. Not sure if it is so good to expect all extra arguments to be datasets. Either kwargs include x, y, z or they should simply not be data arrays.
Sorry for the wrong argument name in the notebook. that wasnt properly updated after the renaming of the argument, as I noted in the old PR, but yeah thats a bit confusing with the old place...

[AdriaanRol]

@MerlinSmiles, I've updated to PyQt5 and the latest version of PyQtGraph and tried this again today as I'm quite eager to improve the live plotting experience (one might call this procrastination).

I still have problems running the version of this branch. This time I get no errors or warnings but just a screen freeze.

Let me give an overview of the combinations that work.

For all variants I run the following snippet

import numpy as np
from qcodes.plots.pyqtgraph import QtPlot
vw = QtPlot(window_title='Title')
x = np.linspace(-10, 10,101)
y=np.cos(x)
vw.add(x=x,y=y)

• QCoDeS master (ca112fe)
  - works fine in ipython kernel
  - works fine in jupyter notebook
• this branch (8846b35)
  - freezes in ipython kernel
  - A plotting window opens that does not update and is not responsive
  - I get the MAC spinning wheel indicating it is loading
  - the kernel returns Out[1]: <Dock initial project plan #1 - Plot (10, 10)>
  - the kernel is responsive
  - freezes in notebook
  - Similar to above but does update the data in the plot
  - Displays the figure inline but is not interactive
  - Also displays a QtPlot window outside of my browser. This displays the line but is also unresponsive.

I'm running

• PyQt5 (5.7.1)
• pyqtgraph (0.10.0)
• python 3.6.0

On a Mac.

I'm curious to know why this is such a problem for me and if there is anything we can change to make your changes take effect. (or am I just behind and are you guys now doing the small chunks in different PR's?)

Best,

[MerlinSmiles]

@AdriaanRol sorry for my late answer.
I don't think that this will be merged anytime soon, as it requires multiprocessing to work.
The problem with the freezing Qt window is that the Qt event scheduler cannot work. In your example if you start a Qt app first with app = pyqtgraph.mkQtApp() or something like that (cant recall exactly) it should not freeze, but you would not have a free terminal, both at the same time on one kernel is not possible.

Also:

• The Qt plot will not be interactive in the notebook, that is not compatible.
  The master works by constantly calling updates, and maybe its still using the pyqtgraph remote something which runs another process by itself but has other disadvantages.
• I wonder how you install pyqt5 with python 3.6 afaik it is not compatible at the moment.
• It does not work in this branch because your config file disables multiprocessing, I think.

[MerlinSmiles]

Closing as this will not work with any recent qcodes version, and is by far superseeded by this one:
https://github.com/MerlinSmiles/Qplot
which gives you an external process for plotting