Fixed large allocations cache

[borisfom]

Only the first diff is non-whitespace only:
my emacs now fixes those extra spaces (flagged later by lint) automatically.
I'd rather keep those whitespace changes than 'clean up' for smaller diff.

[thatguymike]

Also now getting a crash on exit on free. Getting close. I have a local combination of your changes and my changes.

[borisfom]

I am not getting a crash, so that may be a clash with your changes - how do I take a look ?
Try removing debug printouts (now stderr gets mixed with err). Also try a clean rebuild, I don't trust dependencies are 100% complete.

[borisfom]

Re dev_info: no, we are using index into dev info to refer to the device so we need an indirection. The input vector may be "0,3,4" etc.

[borisfom]

Re memory reuse: search for free block is done via lower_bound, and then checking for steam match and bin match. Since the growth factor is 2, then potential memory overuse is 50% for each bin (9K request will in fact always allocate 16as it's rounded to the next pow2 within the bin range (6-16 in our case). All requests larger than the last bin, marked with bin =-1 and allocated exactly (no rounding). This is good part. Bad part is that they act as a single bin, so if 2.5G is the only free block with bin==-1, any other request with bin==-1, say 3M, will take it. With this change I am setting the limit of "wasted" memory to be no more than 1/8 of the request size.

[thatguymike]

Closing in favor of #90 which cleans out whitespace changes and merges this PR and #88