V0

.gitignore

@@ -2,6 +2,9 @@
 __pycache__/
 *.py[cod]
 
+# ipython extras
+.ipynb_checkpoints/
+
 # C extensions
 *.so
 

.sublimelinterrc

@@ -0,0 +1,8 @@
+{
+    "@python": 3.3,
+    "linters": {
+        "pep257": {
+            "excludes": ["*/test_*.py"]
+        }
+    }
+}

objects.md

@@ -0,0 +1,135 @@
+# Object Hierarchy
+
+## Rough linkages:
+
+In **bold** the containing class creates this object.
+In *italics* the container just holds this object (or class) as a default for derivatives to use.
+Normal text the container includes and uses this object
+
+- Station
+  - BaseInstrument: IPInstrument, VisaInstrument, MockInstrument
+    - **Parameter**
+      - Validator: Anything, Strings, Numbers, Ints, Enum, MultiType
+      - **SweepValues**: SweepFixedValues, AdaptiveSweep
+    - Function
+      - Validator
+  - *SweepStorage class*: MergedCSVStorage, ?
+  - **Monitor**
+  - MeasurementSet
+    - StorageManager, Monitor
+    - *SweepStorage class*
+    - .sweep
+      - SweepValues
+      - **SweepStorage**
+  - **StorageManager**
+    - **StorageServer**
+      - SweepStorage
+      - Monitor
+
+## Station
+
+A representation of the entire physical setup.
+
+Lists all the connected `Instrument`s, keeps references to the `StorageManager` and
+`Monitor` objects that are running, and holds various default settings, such as
+the default `MeasurementSet` and the default `SweepStorage` class in use right now.
+
+## Instrument
+
+A representation of one particular piece of hardware.
+
+Lists all `Parameter`s and `Function`s this hardware is capable of, and handles the
+underlying communication with the hardware.
+`BaseInstrument` sets the common structure but real instruments will generally derive
+from its subclasses, such as `IPInstrument` and `VisaInstrument`. There is also
+`MockInstrument` for making simulated instruments, connected to a `Model` that mimics
+a serialized communication channel with an apparatus.
+
+## Parameter
+
+A representation of one particular state variable.
+
+Most `Parameter`s are part of an `Instrument`, but you can also create `Parameter`s
+that execute arbitrary functions, for example to combine several gate voltages in a
+diagonal sweep. Parameters can have setters and/or getters (they must define at least
+a setter OR a getter but do not need to define both)
+
+`Parameter`s can be sliced to create a `SweepFixedValues` object.
+
+## Validator
+
+Defines a set of valid inputs, and tests values against this set
+
+Subclasses include `Anything`, `Strings`, `Numbers`, `Ints`, `Enum`, and `MultiType`,
+each of which takes various arguments that restrict it further.
+
+## SweepValues
+
+An iterator that provides values to a sweep, along with a setter for those values
+connected to a `Parameter`.
+
+Mostly the `SweepFixedValues` subclass is used (this is
+flexible enough to iterate over any arbitrary collection of values, not necessarily
+linearly spaced) but other subclasses can execute adaptive sampling techniques, based
+on the `.feedback` method by which the sweep passes measured values back into the
+`SweepValues` object.
+
+## Function
+
+A representation of some action an `Instrument` can perform, which is not connected to
+any particular state variable (eg calibrate, reset, identify)
+
+## StorageManager
+
+The gateway to the separate storage process.
+
+Sweeps and monitoring routines all pass their data through the `StorageManager` to the
+`StorageServer`, which the `StorageManager` started and is running in a separate process.
+Likewise, plotting & display routines query the `StorageServer` via the `StorageManager`
+to retrieve current data.
+
+## StorageServer
+
+Running in its own process, receives, holds, and returns current sweep and monitor data,
+and writes it to disk (or other storage)
+
+When a sweep is *not* running, the StorageServer also calls the monitor routine itself.
+But when a sweep *is* running, the sweep calls the monitor so that it can avoid conflicts.
+Also while a sweep is running, there are complementary `SweepStorage` objects in the sweep
+and `StorageServer` processes - they are nearly identical objects, but are configured
+differently.
+
+## Monitor
+
+Measures all system parameters periodically and logs these measurements
+
+Not yet implemented, so I don't know the details, but the plan is that during idle times
+it measures parameters one-by-one, keeping track of how long it took to measure each one,
+so that during sweeps it can be given a command "measure for no longer than X seconds" and
+it will figure out which parameters it has time to measure, and which of those are most
+important (primarily, I guess, the most important were last measured the longest time ago)
+
+## MeasurementSet
+
+A list of parameters to measure while sweeping, along with the sweep methods.
+
+Usually created from a `Station` and inheriting the `StorageManager`, `Monitor`,
+and default `SweepStorage` class from it.
+
+The `.sweep` method starts a sweep, which creates a `SweepStorage` object and,
+if the sweep is to run in the background (the default), this method starts the
+sweep process.
+
+## SweepStorage
+
+Object that holds sweep data and knows how to read from and write to disk or
+other long-term storage, as well as how to communicate with its clone on the
+`StorageServer` if one exists.
+
+A `SweepStorage` is created when a new sweep is started, and this one clones
+itself on the `StorageServer`. But you can also create a `SweepStorage` from
+a previous sweep, in which case it simply reads in the sweep and holds it
+for plotting or analysis.
+
+Subclasses (eg `MergedCSVStorage`, later perhaps `AzureStorage` etc) define
+the connection with the particular long-term storage you are using.

qcodes/instrument/base.py

@@ -0,0 +1,141 @@
+import asyncio
+
+from qcodes.utils.metadata import Metadatable
+from qcodes.utils.sync_async import wait_for_async
+from qcodes.instrument.parameter import Parameter
+from qcodes.instrument.function import Function
+
+
+class BaseInstrument(Metadatable):
+    def __init__(self, name, **kwargs):
+        super().__init__(**kwargs)
+        self.functions = {}
+        self.parameters = {}
+
+        self.name = str(name)
+        # TODO: need a sync/async, multiprocessing-friendly lock
+        # should be based on multiprocessing.Lock (or RLock)
+        # but with a non-blocking option for async use
+        # anyway threading.Lock is unpicklable on Windows
+        # self.lock = threading.Lock()
+
+    def add_parameter(self, name, **kwargs):
+        '''
+        binds one Parameter to this instrument.
+
+        instrument subclasses can call this repeatedly in their __init__
+        for every real parameter of the instrument.
+
+        In this sense, parameters are the state variables of the instrument,
+        anything the user can set and/or get
+
+        `name` is how the Parameter will be stored within instrument.parameters
+        and also how you  address it using the shortcut methods:
+        instrument.set(param_name, value) etc.
+
+        see Parameter for the list of kwargs
+        '''
+        if name in self.parameters:
+            raise KeyError('Duplicate parameter name {}'.format(name))
+        self.parameters[name] = Parameter(self, name, **kwargs)
+
+    def add_function(self, name, **kwargs):
+        '''
+        binds one Function to this instrument.
+
+        instrument subclasses can call this repeatedly in their __init__
+        for every real function of the instrument.
+
+        In this sense, functions are actions of the instrument, that typically
+        transcend any one parameter, such as reset, activate, or trigger.
+
+        `name` is how the Function will be stored within instrument.functions
+        and also how you  address it using the shortcut methods:
+        instrument.call(func_name, *args) etc.
+
+        see Function for the list of kwargs
+        '''
+        if name in self.functions:
+            raise KeyError('Duplicate function name {}'.format(name))
+        self.functions[name] = Function(self, name, **kwargs)
+
+    def snapshot_base(self):
+        return {
+            'parameters': dict((name, param.snapshot())
+                               for name, param in self.parameters.items()),
+            'functions': dict((name, func.snapshot())
+                              for name, func in self.functions.items())
+        }
+
+    ##########################################################################
+    # write, read, and ask are the interface to hardware                     #
+    #                                                                        #
+    # at least one (sync or async) of each pair should be overridden by a    #
+    # subclass. These defaults simply convert between sync and async if only #
+    # one is defined, but raise an error if neither is.                      #
+    ##########################################################################
+
+    def write(self, cmd):
+        wait_for_async(self.write_async, cmd)
+
+    @asyncio.coroutine
+    def write_async(self, cmd):
+        # check if the paired function is still from the base class (so we'd
+        # have a recursion loop) notice that we only have to do this in one
+        # of the pair, because the other will call this one.
+        if self.write.__func__ is BaseInstrument.write:
+            raise NotImplementedError(
+                'instrument {} has no write method defined'.format(self.name))
+        self.write(cmd)
+
+    def read(self):
+        return wait_for_async(self.read_async)
+
+    @asyncio.coroutine
+    def read_async(self):
+        if self.read.__func__ is BaseInstrument.read:
+            raise NotImplementedError(
+                'instrument {} has no read method defined'.format(self.name))
+        return self.read()
+
+    def ask(self, cmd):
+        return wait_for_async(self.ask_async, cmd)
+
+    @asyncio.coroutine
+    def ask_async(self, cmd):
+        if self.ask.__func__ is BaseInstrument.ask:
+            raise NotImplementedError(
+                'instrument {} has no ask method defined'.format(self.name))
+        return self.ask(cmd)
+
+    ##########################################################################
+    # shortcuts to parameters & setters & getters                            #
+    #                                                                        #
+    #  instrument['someparam'] === instrument.parameters['someparam']        #
+    #  instrument.get('someparam') === instrument['someparam'].get()         #
+    #  etc...                                                                #
+    ##########################################################################
+
+    def __getitem__(self, key):
+        return self.parameters[key]
+
+    def set(self, param_name, value):
+        self.parameters[param_name].set(value)
+
+    @asyncio.coroutine
+    def set_async(self, param_name, value):
+        yield from self.parameters[param_name].set_async(value)
+
+    def get(self, param_name):
+        return self.parameters[param_name].get()
+
+    @asyncio.coroutine
+    def get_async(self, param_name):
+        return (yield from self.parameters[param_name].get_async())
+
+    def call(self, func_name, *args):
+        return self.functions[func_name].call(*args)
+
+    @asyncio.coroutine
+    def call_async(self, func_name, *args):
+        return (yield from self.functions[func_name].call_async(*args))

qcodes/instrument/function.py

@@ -0,0 +1,75 @@
+import asyncio
+
+from qcodes.utils.metadata import Metadatable
+from qcodes.utils.sync_async import syncable_command
+from qcodes.utils.validators import Validator
+
+
+class Function(Metadatable):
+    def __init__(self, instrument, name, call_cmd=None, async_call_cmd=None,
+                 parameters=[], parse_function=None, **kwargs):
+        '''
+        defines a function (with arbitrary parameters) that this instrument
+        can execute.
+
+        instrument: an instrument that handles this function
+        name: the local name of this parameter
+        call_cmd: command to execute on instrument
+            - a string (with positional fields to .format, "{}" or "{0}" etc)
+            - a function (with parameter count matching parameters list)
+        async_call_cmd: an async function to use for call_async, or for both
+            sync and async if call_cmd is missing or None.
+        parameters: list of Validator objects,
+            one for each parameter to the Function
+        parse_function: function to parse the return value of cmd,
+            may be a type casting function like int or float.
+            If None, will not wait for or read any response
+        '''
+        super().__init__(**kwargs)
+
+        self._instrument = instrument
+        self.name = name
+
+        self._set_params(parameters)
+        self._set_call(call_cmd, async_call_cmd, parse_function)
+
+    def _set_params(self, parameters):
+        for param in parameters:
+            if not isinstance(param, Validator):
+                raise TypeError('all parameters must be Validator objects')
+        self._parameters = parameters
+        self._param_count = len(parameters)
+
+    def _set_call(self, call_cmd, async_call_cmd, parse_function):
+        ask_or_write = self._instrument.write
+        ask_or_write_async = self._instrument.write_async
+        if isinstance(call_cmd, str) and parse_function:
+            ask_or_write = self._instrument.ask
+            ask_or_write_async = self._instrument.ask_async
+
+        self._call, self._call_async = syncable_command(
+            self._param_count, call_cmd, async_call_cmd,
+            ask_or_write, ask_or_write_async, parse_function)
+
+    def validate(self, args):
+        if len(args) != self._param_count:
+            raise TypeError(
+                '{} called with {} parameters but requires {}'.format(
+                    self.name, len(args), self._param_count))
+
+        for i in range(self._param_count):
+            value = args[i]
+            param = self._parameters[i]
+            if not param.is_valid(value):
+                raise ValueError(
+                    '{} is not a valid value for parameter {} of {}'.format(
+                        value, i, self.name))
+
+    def call(self, *args):
+        self.validate(args)
+        return self._call(*args)
+
+    @asyncio.coroutine
+    def call_async(self, *args):
+        self.validate(args)
+        return (yield from self._call_async(*args))