Optimized accuracy calculation

[Noiredd]

Note: sorry about potential confusion arising from multiple edits to this PR. It is complete now, unless someone finds room for more improvement.

Motivation: accuracy calculation, particularly for segmentation tasks with top_k: 1, can be extremely slow: note the two nested for loops (over the batch and over image pixels) with partial_sort inside. The main culprit here is the need to copy all data to a new container so we can sort it.

My proposal is to replace that with a dynamically updated priority_queue: instead of copying everything and thinking later, we can iterate just once and only copy an element when it's larger than the smallest of k currently best ones (provided by an automatically sorted container).
Initially I thought of having a separate case for top_k: 1, where we would only make a single iteration and remember the single best element, but it is only marginally faster than the queue approach (see below), therefore I abandoned the idea for the sake of code clarity.

Benchmark settings:

• net: FCN_AlexNet, batch size 1,
• dataset: PASCAL_VOC (21 classes, 1464 images in training set, 1449 for validation, images are about 360x550) - about 250M top_k searches per epoch,
• solver setup: max_iter: 14640, test_interval: 1464, test_iter: 1449,
• hardware: Titan Z, 2x Xeon E5620 (2.4 GHz),
• software: Ubuntu 14.04, CUDA 8.0, cuDNN 5.1, ATLAS, DIGITS 5.1-dev.

Each build was ran 4 times (Titan Z can run two nets in parallel so it was effectively 2 times 2 runs), times were measured from job initialization to completion (as reported by DIGITS).

Results:

top_k	current master	priority_queue	optimal search
1	90m 00s	58m 57s	58m 34s
5	98m 49s	88m 42s	N/A

For me this is about 10% faster for top-5 accuracy, and over 30% faster for top-1.

Task list:

• optimize the top_k==1 case
• optimize the general case
• provide a simple benchmark
• simplify to an insignificantly slower but much clearer approach.

[Noiredd]

Is there any way I can get to the details of the Travis build error? My make runtest passed without problems, but there I got a "100% mismatch", which seems confusing because the visible parts of x and y arrays look identical.

EDIT: this is not the only PR to fail for this reason. #5595 resulted in a similar line, #5558 as well, same with the current state of #5676 (log).

[Noiredd]

I guess I force-pushed too much and broke Travis. Going to close this and open a new PR.