Improve the exception when attempting to partially collapse a multidimensional coordinate

[pelson]

Example:

import iris.tests.stock as stock

c = stock.simple_4d_with_hybrid_height()
print c

print c.collapsed('grid_latitude', iris.analysis.MEAN)

Output:

air_temperature                     (time: 3; model_level_number: 4; grid_latitude: 5; grid_longitude: 6)
     Dimension coordinates:
          time                           x                      -                 -                  -
          model_level_number             -                      x                 -                  -
          grid_latitude                  -                      -                 x                  -
          grid_longitude                 -                      -                 -                  x
     Auxiliary coordinates:
          level_height                   -                      x                 -                  -
          sigma                          -                      x                 -                  -
          surface_altitude               -                      -                 x                  x
     Derived coordinates:
          altitude                       -                      x                 x                  x
iris/coords.py:673: UserWarning: Collapsing a non-contiguous coordinate. Metadata may not be fully descriptive for "grid_latitude".
  'not be fully descriptive for "%s".' % self.name())
Traceback (most recent call last):
  File "example.py", line 26, in <module>
    print c.collapsed('grid_latitude', iris.analysis.MEAN)
  File "iris/cube.py", line 1849, in collapsed
    collapsed_cube.replace_coord(coord.collapsed(local_dims))
  File "iris/coords.py", line 665, in collapsed
    raise ValueError('Cannot partially collapse a coordinate (%s).' % self.name())
ValueError: Cannot partially collapse a coordinate (surface_altitude).

Consider being explicit about removing the coordinate (and an example?) in the actual exception.

This issue is a direct result of a question I was asked offline by a frequent user of Iris.

[duncanwp]

We've also hit this problem so I'm interested in your solution (and example!). Are you implying we have to remove the auxiliary coordinates before collapsing?

Are there any plans for being able to collapse auxiliary coordinates in the future?

Thanks

[pelson]

Are there any plans for being able to collapse auxiliary coordinates in the future?

There is nothing in the pipeline AFAIK.

We've also hit this problem so I'm interested in your solution (and example!). Are you implying we have to remove the auxiliary coordinates before collapsing

Yes - cube.remove_coord (http://scitools.org.uk/iris/docs/latest/iris/iris/cube.html?highlight=remove#iris.cube.Cube.remove_coord) should be good enough to work around the issue.

[duncanwp]

OK, so in general we could loop over all those auxiliary coordinates with dimensions greater than one and remove them. Sorry, I could check the code, but would I also have to remove derived coordinates?

[duncanwp]

It's OK, I've convinced myself you would have to - since the derived coordinate would be derived from one of the auxiliaries!

[ajdawson]

so in general we could loop over all those auxiliary coordinates with dimensions greater than one and remove them

Unless all or none the dimensions they span are to be collapsed... I think that is how it works.

[duncanwp]

Yeah OK, that makes sense.

I'm a little surprised this isn't implemented, I guess usually if you're doing a zonal mean (for example) you will have collapsed the altitude component.

Presumably it would be pretty straightforward to just collapse the offending coordinate in the same way the data is? Perhaps with a warning that this is happening?

[duncanwp]

I'm looking to implement this in CIS, but it would be nicer within Iris I think - if it fits. I've taken a quick look and it seems like passing an aggregator to coord.collapse() would be one way of doing this.

Here's a very rough first stab just to give you an idea of my proposed approach. If you think it would be useful I'll carry on, but if not I'll just work around it in CIS!

duncanwp@4f1e299

[pelson]

That is a neat idea @duncanwp. What would you do with the aggregated coordinates though? Does your workflow look something like:

 * get the coordinate which is problematic
 * remove it from the cube
 * apply the aggregation on the cube
 * apply the aggregation on the coordinate
 * add the resulting coordinate onto the resulting cube

? Or have I completely mis-understood?

[duncanwp]

That's exactly right. I needed something working quickly so I've actually implemented this myself in CIS: https://github.com/cedadev/cis/blob/master/cis/aggregation/aggregator.py#L111 (and associated methods).

It works quite nicely, but one of the remaining difficulties is that Iris (or any other tool for that matter) has no way of knowing that the Aux coordinate used be a function of that collapsed dimension, so you can't easily merge cubes back together having aggregated them. For cubes you use scalar coordinates (and cell methods) to identify this, but it wasn't immediately obvious how you might do that for a multi-dimensional coordinate.

It's not a big deal, and perhaps it's useful as it is, in which case I'll happily put together a pull request.

[DPeterK]

There's a note about this in the docs now; see #1819.

[rcomer]

Since #3028 we now can collapse multidimensional auxcoords, so I think this issue could be closed. However,

There's a note about this in the docs now; see #1819.

That note is still in the User Guide, so we should probably take it out now.