ReductionLayer

[jeffdonahue]

Performs a "reduction" operation (currently SUM, MEAN, ASUM for sum of absolute values, and SUMSQ for sum of squares) to turn a number of "tail" axes into a single scalar value. The MEAN operation, in combination with a loss_weight, is useful for creating custom losses that don't have an obnoxious amount of output. For example, this EuclideanLoss:

layer {
  type: "EuclideanLoss"
  bottom: "preds"
  bottom: "targets"
  top: "mean_squared_error"
  loss_weight: 1
}

...is equivalent to this Reduction:

layer {
  type: "Eltwise"
  bottom: "preds"
  bottom: "targets"
  top: "error"
  eltwise_param {
    operation: SUM
    coeff: 1
    coeff: -1
  }
}
layer {
  type: "Reduction"
  bottom: "error"
  top: "sum_squared_error_per_instance"
  reduction_param { operation: SUMSQ axis: 1 }
}
layer {
  type: "Reduction"
  bottom: "sum_squared_error_per_instance"
  top: "mean_squared_error"
  reduction_param { operation: MEAN axis: 0 coeff: 0.5 }
  loss_weight: 1
}

(would be more efficient to do as a single Reduction with SUM_OF_SQUARES and a certain coeff setting, but then you have to compute the batch size and everything is less pretty...)

Eventually, this should support reduction along inner axes (e.g. support an end_axis), but that makes the implementation substantially more involved than this...

[shelhamer]

SUM_OF_SQUARES is a little unlike the other op names. SUMSQ fits in with ASUM. Your pick.

[jeffdonahue]

fair enough -- I'll change it to SUMSQ

[shelhamer]

Once the comments are addressed this looks fine, although there could be a test for reducing over a tail that isn't all dimensions.

p.s. N-D blobs are nice. count() and CanonicalAxisIndex()make for more obvious code.

[jeffdonahue]

Thanks for all the reviews @shelhamer! Tests for non-zero axis added; will merge after Travis.

[jeffdonahue]

Wow... especially thanks for the suggestion to test other axes. I had a bug in Backward for asum and sumsq (wasn't incrementing bottom_data). Should be good now...

[shelhamer]

Tests are more trustworthy than me. Good catch!