V0

[alexcjohnson]

@qdev-dk/qcodes here's a first cut framework. (esp. @alan-geller @akhmerov @guenp @spauka - apologies for the long time since I said "it's almost here!")

The example .ipynb should be self-contained (but you have to change qcpath in the first cell to wherever you put the package when you clone it) - it defines a few mock instruments, makes an experiment that connects them to a toy model, and runs a couple of sweeps. The sweeps run (by default) in a separate process, and you can refresh their plots when you want (making this automatic is one of the next steps).

I haven't yet made any real instrument drivers (I still need to play with sync/async visa and IP commands) but they'll look pretty much like the mock instruments in the notebook, with a few available additions like get_cmd etc can be arbitrary functions in addition to just strings. I also haven't done anything about monitoring, but I'll point out where I expect it to go.

I'm most interested in issues about structure - the kind of objects we're making and how they relate to each other - but any and all comments & questions are welcome, either general or connected to specific lines of the code. (and as before, don't merge it, I'll do that when we're all happy).

[alexcjohnson]

I'm hoping that async commands will work well for most of the instrument drivers, but that will take some experimentation. Regardless, this set of wrappers will let people define whichever version they can for any given instrument, and sweeps can run either sync or async, making the best of what they find.

[alexcjohnson]

Also, I started with the snazzy new python 3.5 syntax but ended up downgrading to 3.3 because anaconda (and my linter) are still going to take a while to play nicely with the new syntax.

[akhmerov]

Hey Alex, congrats on putting this together!

I get this when running the example notebook on 3.4.3:

...Qcodes/qcodes/utils/validators.py in Numbers()
     84     '''
     85 
---> 86     def __init__(self, min_value=-math.inf, max_value=math.inf):
     87         if isinstance(min_value, (float, int)):
     88             self._min_value = min_value

AttributeError: 'module' object has no attribute 'inf'

EDIT: nvm @guenp already got this.

[dbwz8]

Change all the math.* things to float(‘*’) (math.nan -> float(‘nan’).

Python 3.5 syntax item.

From: Anton Akhmerov [mailto:notifications@github.com]
Sent: Wednesday, October 14, 2015 12:37 PM
To: qdev-dk/Qcodes Qcodes@noreply.github.com
Cc: Dave Wecker dbwz8@hotmail.com
Subject: Re: [Qcodes] V0 (#2)

Hey Alex, congrats on putting this together!

I get this when running the example notebook on 3.4.3:

...Qcodes/qcodes/utils/validators.py in Numbers()
84 '''
85
---> 86 def init(self, min_value=-math.inf, max_value=math.inf):
87 if isinstance(min_value, (float, int)):
88 self._min_value = min_value

AttributeError: 'module' object has no attribute 'inf'

—
Reply to this email directly or view it on GitHub qdev-dk-archive#2 (comment) . https://github.com/notifications/beacon/AKEAqWZ0s6dttfHUDGGSvIHTsNYHT99Fks5o7alRgaJpZM4GOUO0.gif

[akhmerov]

OK, everything runs now.

[guenp]

(not sure if this is the right place for this comment, but it applies to the Qcodes example.ipynb file)
when I run the following cell in the notebook:

 swp.sync_live()
 plt.plot(swp['chan0'], swp['amplitude'])

Somehow, when re-running the cell, it requires me to rerun %matplotlib nbagg, otherwise the figure will not display and just returns

[<matplotlib.lines.Line2D at 0x10af29c50>]

Using: Chrome, OSX yosemite

edit: this is not the case for the second example (pcolormesh)

[akhmerov]

I believe it's a known nbagg deficiency. You can also "fix" it by running either plt.close() or plt.show(), I don't remember which.

[akhmerov]

@alexcjohnson , can you perhaps add a brief document giving an overview of the object hierarchy? Everything looks relatively straightforward, but I don't want to be making wrong assumptions. Also I think that would be a good location for the global design discussion.

[guenp]

Loving the code so far, great job @alexcjohnson! 🙌

I'll have more comments as I'll go through & test it later this week but here's some first thoughts:

• I think the 'measurement' object is an awesome idea, to dynamically add measured parameters - this saves a lot of time for large sweeps where you don't want to measure static parameters every datapoint. However, for 2D or 3D sweeps you might want to monitor quasi-static parameters every n datapoints (let's say, after every inner loop). Perhaps the measurement object could have a list of parameters to measure for every sweep dimension?
• Parameters should also have the keyword argument 'units' e.g. V(olts), T(esla), etc. and upper/lower limits (that could be dynamic and read out from the instrument, e.g. compliance limits)
• I would like to suggest to add an extra layer of abstraction there for those who don't want to think about meter['amplitude'] but rather about a physical parameter such as device1.drain_voltage (but perhaps this is already on the todo-list)
• When I talked to @akhmerov last time, we discussed using the python @property syntax for instrument parameters, which I've played around with during my last measurement run. Perhaps it's a matter of preference, but I like the idea of being able to run instrument.channel instead of instrument[channel].get() or instrument.channel = x instead of instrument[channel].set(x). This could also be experiment-specific e.g. experiment.backgate_voltage = 5 which would run decadac['chan0'].set(5)
• I assume live plotting is still on the todo-list, but imho pretty important for actual usage. when measuring you'll want to see data live while it's coming out.
• I haven't tried it but is it possible to set instrument parameters while a sweep is running?
• I'm pretty curious to hear about your experience with Azure so far 😃 I'm still running into the issue of what to do with my data files. They're on dropbox so far but to me that doesn't seem ideal... of course local storage is essential in case of network failures etc. but in standard conditions I'd much rather work from the cloud since my data analysis is on a laptop anyway.

Cheers,
Guen

[alexcjohnson]

@guenp lots of pieces to respond to, thanks!

However, for 2D or 3D sweeps you might want to monitor quasi-static parameters every n datapoints (let's say, after every inner loop). Perhaps the measurement object could have a list of parameters to measure for every sweep dimension?

Interesting - do you have a particular use case in mind? Or would this be taken care of by the monitor framework (that I haven't written, but discussed above). In that concept this data isn't connected to the sweep data by any means aside from the timestamp, but are there cases that you'd like it to be an integral part of the sweep?

Parameters should also have the keyword argument 'units' e.g. V(olts), T(esla), etc. and upper/lower limits (that could be dynamic and read out from the instrument, e.g. compliance limits)

👍 - the parameters do have limits (Numbers(-10,10) etc) but a softer limit setting that's easier to change on the fly is a good idea.

I would like to suggest to add an extra layer of abstraction there for those who don't want to think about meter['amplitude'] but rather about a physical parameter such as device1.drain_voltage (but perhaps this is already on the todo-list)

This would be pretty easy for a user to set up on their own, not sure how much I could do to facilitate that but I'll take a look.

When I talked to @akhmerov last time, we discussed using the python @property syntax for instrument parameters, which I've played around with during my last measurement run. Perhaps it's a matter of preference, but I like the idea of being able to run instrument.channel instead of instrument[channel].get() or instrument.channel = x instead of instrument[channel].set(x). This could also be experiment-specific e.g. experiment.backgate_voltage = 5 which would run decadac['chan0'].set(5)

That should work for setting/getting individual values, but it makes it a little tricky to pass parameters into a sweep. Perhaps though there is a way to keep the syntax distinct so we can have it both ways.

I assume live plotting is still on the todo-list, but imho pretty important for actual usage. when measuring you'll want to see data live while it's coming out.

Yup - that's high on the list. Now that the sweep itself is running in a separate process, feeding the data to yet another process, this should be as easy as wrapping the plotting calls up in a SweepStorage method that sends some javascript to the notebook to keep refreshing the plot until the sweep is done.

I haven't tried it but is it possible to set instrument parameters while a sweep is running?

I haven't yet found a Lock that's built to work with async code, sync code, and multiple processes, I may have to write one. Until then, it should generally work but may be unsafe to set parameters on an instrument that's also involved in the sweep.

I'm pretty curious to hear about your experience with Azure so far 😃 I'm still running into the issue of what to do with my data files. They're on dropbox so far but to me that doesn't seem ideal... of course local storage is essential in case of network failures etc. but in standard conditions I'd much rather work from the cloud since my data analysis is on a laptop anyway.

I actually haven't played with Azure yet, but I agree that it's the direction to try.

[akhmerov]

I actually haven't played with Azure yet, but I agree that it's the direction to try.

I missed the Azure discussion, what's the story there?

[guenp]

@alexcjohnson thanks for the response!
On the quasi-static parameters: Yes, the monitor would probably solve this issue (and I think it's great if you figure out a way to monitor as many parameters as possible without sacrificing measurement time), however to give an example, let's say you are doing a 2D Current-B-field sweep. At every B-field, you sweep the current and measure the voltage. Let's say that your sample heats up a little bit at every B-field step and you want to keep track of this. Then in the inner loop I'd want to step current, measure voltage, and in the outer loop I'd want to step B-field and measure temperature.

@akhmerov Azure is the Microsoft solution to cloud storage. The idea would be to have a data storage process send datapoints to the cloud directly while taking data (provided the datapoints are also stored locally & there's a working internet connection). Then you could just import data in your ipython notebook/matlab/etc. online for post-analysis.

[alexcjohnson]

Maybe I'm missing something, but I'm still fairly convinced that Monitor needs to be an independent, unique object, rather than a MeasurementSet instance, and not really part of a sweep, but called by the sweep, in such a way that the sweep is in full control.

As @akhmerov mentions, it can do more than just measure values, it can act on those measurements (either feedback control or error response). But during a sweep, the main acquisition task is king. I'm not sure quite what you had in mind regarding mixed blocking/nonblocking, but I was envisioning treating all measurements the monitor wants to do as blocking, at least during sweeps. What I have in mind is roughly:

• the monitor holds a list of parameters to measure, and a list of actions to perform based on those measurements. Perhaps also a priority and/or target frequency for each of the parameters.
• as time passes, it learns statistics about how long it takes to measure (and act on) each parameter
• when not sweeping, monitor basically has free reign to measure
• during sweeps, the sweep gives the monitor a time it must try very hard not to exceed, and it measures the most pressing parameters it's confident it can in that time.
• if a sweep parameter needs monitor data to tune its own values, these are accessible by querying the StorageManager. The status pane, when we get to that, will also just query for these values.
• the user should be able to change what's monitored, and what conditions will trigger what actions, at any time
• in general we should be logging all of these measurements, forever. Yes, this may make some large files, that's OK, it's not going to be terabytes. Perhaps we can set a low frequency (10 min? 1 hr?) and store the data in open/high/low/close form so we don't lose spikes.

[alexcjohnson]

Why is there a need for a separate Function class, as opposed to making it an instrument method?

you mean instead of inst.functions['reset'].call() just have inst.reset()? I like the idea of creating such shortcuts (individual drivers can certainly do that), but I do think there's enough extra built into Function - sync/async and parameter validation - that it makes sense to keep it. Also it's nice to be able to list them, without getting functions and other methods mixed up together.

[alexcjohnson]

Metadatable and Validator look similar to traitlets. Would using traitlets directly make sense?

Possibly! As already mentioned, I'm not sure if Metadatable is going to live on, but if it could be replaced with a built-in that would be handy. I'll look at whether traitlets can do what we need for Validator.

[alexcjohnson]

@akhmerov re: Storage format:

You're quite right about the two parts, the explicit measurements vs the log info. The latter is primarily the responsibility of the monitor but we may also want to snapshot it along with each sweep, so that you can look up the full system state during a sweep without having to correlate timestamps and hunt through a log file.

re: the best format for each sweep type - totally agreed, and it also depends on the size of the sweep, the number of things being measured simultaneously, and personal / team preferences. I put together MergedCSVStorage as basically an example, expecting that we will make many more as we go along, but each sweep can specify its own storage_class for exactly this reason. The question then becomes, how do we figure out what format some data is in when we want to load it back in? Perhaps that is another function of the log snapshot file.

[alexcjohnson]

re: Calculation and Human - 👍
This brings up though the question of how to handle raw and processed data - we could shoehorn the processed data into Parameter objects (with some clever connection to the raw Parameter(s) so we don't end up measuring twice, and we ensure that the calculation doesn't happen until the data has arrived) but it might be worth making a separate category within the MeasurementSet that would handle these constraints more naturally - perhaps in fact we can make an object that lists the raw values it needs in such a way that the MeasurementSet pulls them in - eg you say you want to measure R_xx and it instructs the MeasurementSet to measure I_x and V_x... then if it's really smart, you can ask for R_xy also and it will know to only measure I_x once but add in V_y

[akhmerov]

@guenp @alexcjohnson re: storage format

Isn't this essentially the same as separating the Monitor from the MeasurementSet/sweep?

Not exactly: as Alex also notes, the full log should also contain the data from the instruments involved in the measurement, so that the full history of experiment is recorded.

This can be handled by the StorageManager. It should be capable of archiving old/large data files and logs while still keeping them accessible to the user at any time.

There are cases when the raw data takes unreasonably large volume. Imaine taking raw data at megasample/sec rate when all you need is accurate statistics (averages and errors). You'll have terabytes to story in a matter of days.

Maybe I'm missing something, but I'm still fairly convinced that Monitor needs to be an independent, unique object, rather than a MeasurementSet instance, and not really part of a sweep, but called by the sweep, in such a way that the sweep is in full control.

Here's how I imagine it:

You see a Scheduler responsible for deciding what to measure at which moment. User can update the scheduler strategy on the fly by adding or removing measurements, monitor activities, or updating the logic. Scheduler is responsible for composing the full measurement log since it has all the information. Measurements define their own "processed" storage format, that is kept separate from the full log. Scheduler may use all the information in further planning. It is also aware about what can and what cannot be done in parallel, something that's hard to incorporate into the hard separation between Sweep and Monitor. One use of Monitor that also isn't fitting the paradigm of it being non-blocking is emergency shutdown.

The problem of course, is that all the complexity now is hidden in the definition of sheduling priorities and the solving of the scheduling problem. Nonetheless, simple workflows are easy to incorporate.

[akhmerov]

@alexcjohnson re: Function

I do think there's enough extra built into Function - sync/async and parameter validation - that it makes sense to keep it.

Methods can be as fancy as it gets, including validation and asynchronicity. I just think that it makes sense to keep the exposed interface simple and elegant.

[akhmerov]

I have a workflow suggestion: the code in this PR fulfils its purpose. Shall we merge, and instead separate the discussion into issues?

[guenp]

Maybe I'm missing something, but I'm still fairly convinced that Monitor needs to be an independent, unique object, rather than a MeasurementSet instance, and not really part of a sweep, but called by the sweep, in such a way that the sweep is in full control.

I'm going to assume first that by 'sweep' you mean a 'measurement'.
I don't think the monitor should be part of a measurement - it should run in parallel to a measurement.
And I don't think it should be called by the sweep but rather should be running in the background all the time. Otherwise you would all of a sudden stop monitoring if a measurement ends.

As @akhmerov mentions, it can do more than just measure values, it can act on those measurements (either feedback control or error response).

A measurement should be able to do the same. (e.g. I've done an experiment once where I tuned the temperature while sweeping the magnetic field such that my sample resistivity stayed at a certain value.)

But during a sweep, the main acquisition task is king.

This depends on the priority you assign to a measurement process. I think the Scheduler should be king and decides which process gets instrument communication dibs.

I'm not sure quite what you had in mind regarding mixed blocking/nonblocking, but I was envisioning treating all measurements the monitor wants to do as blocking, at least during sweeps.

It shouldn't have to be, if instruments are connected via different ports. The PC should be able to handle talking to GPIB and a COM port in parallel, however, all GPIB communication should be blocking as instruments are daisy-chained together to one port.

• as time passes, it learns statistics about how long it takes to measure (and act on) each parameter

Machine learning? Ambitious 👍

• when not sweeping, monitor basically has free reign to measure
• during sweeps, the sweep gives the monitor a time it must try very hard not to exceed, and it measures the most pressing parameters it's confident it can in that time.

I still think the Scheduler should delegate, not the measurement process.

• if a sweep parameter needs monitor data to tune its own values, these are accessible by querying the StorageManager.

That's a bit confusing to me. Why not directly talk to the monitor or read the monitor's data log/DataSet? Perhaps I don't understand the purpose you intended for StorageManager.

• the user should be able to change what's monitored, and what conditions will trigger what actions, at any time

This makes it look more and more like a measurement (hence my suggestion to make it a subclass of MeasurementSet.)

• in general we should be logging all of these measurements, forever. Yes, this may make some large files, that's OK, it's not going to be terabytes. Perhaps we can set a low frequency (10 min? 1 hr?) and store the data in open/high/low/close form so we don't lose spikes.

As I said earlier I suggest to make this a temp buffer

[guenp]

@akhmerov

Not exactly: as Alex also notes, the full log should also contain the data from the instruments involved in the measurement, so that the full history of experiment is recorded.

That sounds redundant to me, can't the monitor just refer to the dataset instead and just log parameters that aren't included in the measurement?

There are cases when the raw data takes unreasonably large volume. Imaine taking raw data at megasample/sec rate when all you need is accurate statistics (averages and errors). You'll have terabytes to story in a matter of days.

Sure, the StorageManager can then decide to omit these files after post-analysis is completer, or indicated by the user

Here's how I imagine it:

What exactly would the 'Full log' look like in your opinion? Should it include information such as when measurements were started, and which processes are given priority at which time? In that case I think that should be a separate log (i.e. Scheduler log) :) The monitor is just there such that you can retrace the status of the system at any given time and for the user to see what the instruments values are while the system is taking data.

Btw, there's no reason why any parameter that's being recorded for a measurement shouldn't be updated in the monitor at the same time (e.g. if the measurement probes the temperature, the monitor could also save that as a datapoint simultaneously)

[guenp]

I have a workflow suggestion: the code in this PR fulfils its purpose. Shall we merge, and instead separate the discussion into issues?

Agree! I'm starting to lose track of what we've agreed on or not, and what's still up for discussion.

[akhmerov]

Merging (@guenp and I think this PR is merge-ready). Here's a 💃 for @alexcjohnson :-)