new OffHeapIncrementalIndex that does only aggregations off-heap

[himanshug]

Our groupBy queries contain many metric columns of type thetaSketch which are big and lead to significant GC pressure on the process, this PR enables doing metric aggregation for group by queries completely off-heap.

changes introduced:

1. new OffHeapIncrementalIndex that is same as OnHeapIncrementalIndex but does aggregations in off-heap buffers.
2. configuration, "useOffheap", to use OffHeapIncrementalIndex while GroupBy merging
3. configuration, "druid.computation.buffer.pool.cacheSize", to set the upper limit on number of buffers that StupidPool can cache

fixes #2297

I understand that this PR would have conflicts with some other open PRs. I will rebase and fix merge conflicts as and when other PRs get merged.

[fjy]

@himanshug have you thought at all about extending this PR and enabling on disk computations with groupBys?

[himanshug]

@fjy This is a self-sufficient step that has immediate gains without introducing any major query latency.

I'm thinking about on-disk, but it is a challenge to do that without adding significant query latency... currently, I'm in the stage of brainstorming on that in the background. That said, current PR might enable larger groupBys than before.

[nishantmonu51]

@himanshug @fjy , does this need to be a blocker for 0.9.0 ?

[drcrallen]

@himanshug Have you guys ever tried enabling swap and using more direct byte buffers than available memory?

[drcrallen]

can this just move to a protected class instead of duplicating it?

[himanshug]

will do

[himanshug]

@drcrallen thanks for review, I will probably rework this a bit after #2085 is merged.
we have very high memory brokers and don't usually need swap at all. this PR is intended to avoid GC pauses really. at historicals, we have some subsets where more segments than available memory are loaded, but we have SSDs there and it works ok.

[himanshug]

also, I do not want to block 0.9.0 RC because of this so setting the milestone for this to be 0.9.1
However, I will pull it in my internal distribution as soon as it is merged.

[himanshug]

@drcrallen all review comments addressed.

[fjy]

👍

[drcrallen]

That's not quite true. If I have a max size of 10 and I request 100 buffers, I'll still get 100 buffers, but I'll only return 10 for later use. Then if I request 100 I'll have 10 queued and 90 new ones. And when they are done they will return 10 and drop 90 to GC.

[drcrallen]

I'm at a loss right now on the proper way to document that behavior. Maybe just in the stupidPool javadoc?

[himanshug]

yes, you are right and this is just size of the cache. do you want to suggest better messaging?

[himanshug]

how about, "processing buffer pool caches the buffers for later use, this is the maximum count cache will grow to. however pool can create more buffers than it can cache."

[himanshug]

ok updated doc to be more explicit.

[drcrallen]

FYI there's io.druid.segment.CloserRule for just this kind of use

[himanshug]

that looks nicer... changed to use CloserRule.

[himanshug]

@drcrallen addressed comments.

[drcrallen]

Cool 👍 with 4 commits (after travis) @himanshug would you be willing to update this thread with performance numbers between onheap and offheap after your internal testing?

(I'm assuming that would come after this is merged)

[himanshug]

@drcrallen thanks, will update with test results, thatz why I haven't made the off-heap default but opt-in.