Move from ByteBuffer to Memory

[leventov]

The goal of this issue is to agree on basic design principles and expectations from this refactoring.

Goals

• Speed up critical places in query processing by skipping bound checks, which ByteBuffer always performs when making any read or write. Bound checks could be hoisted outside the hot loops manually or enabled via assertions/static final boolean flag, that is effectively free with some precautions.
• Get rid or 2 GB limitation, use long indexes. See Removing Integer.MAX column size limit. #3743.
• Don't make a lot of Object copies when a Memory object has to be "sliced", "converted to read-only", etc. See define new high-level abstract storage interfaces with a default mmap implementation #3716 (comment).
• + Eliminate current Endian craziness in Druid by standardizing on native-endian. Move from ByteBuffer to Memory #3892 (comment)

Design

• Based on DataSketches's Memory.
• It is moved to Druid source tree, not used as a dependency. See Move from ByteBuffer to Memory #3892 (comment)
• Memory object is immutable. "position and limit", if needed, are passed along with Memory as two primitive longs.
• Upon creation a mutable Memory object has a cached immutable "view" object (which implements read methods and throws exception from write methods), this object is always returned when "as read only" is needed.
• Explicit close() or free() on Memory is possible, but not strictly required, there is a sun.misc.Cleaner safety net, as well as in DirectByteBuffer.
• While transition from ByteBuffer to Memory is in progress (not expected to be done all at once, but subsystem by subsystem, class by class see Move from ByteBuffer to Memory #3892 (comment)), and also when we need to interop with external libraries which require ByteBuffer, conversion from Memory to ByteBuffer and vice versa is possible. Likely it requires to
  • Use DirectByteBuffer-compatible format of Cleaner inside Memory
  • Be able to access private DirectByteBuffer constructors via MagicAccessorImpl.
• + "Global" (immutable) Memory's bounds are checked optionally via assertions/guarded by static final boolean flag, "local" position and limit are check explicitly manually, with helper methods or versions of read and write methods of Memory, which accept read/write position and "local" limits. Move from ByteBuffer to Memory #3892 (comment)

Objections, additions, corrections, questions are welcome.
@leerho @cheddar @weijietong @akashdw @himanshug @fjy @niketh

[leventov]

Unresolved problems

• Big-endian format that is currently used in Druid and ByteBuffers?

[leventov]

+ Design

• "Global" (immutable)Memory's bounds are checked optionally via assertions/guarded by static final boolean flag, "local" position and limit are check explicitly manually, with helper methods or versions of read and write methods of Memory, which accept read/write position and "local" limits.

[cheddar]

@leventov there's a problem with moving Memory to the Druid source tree, which is that it would then make the current implementations of Sketches not work. They would have to either be re-implemented depending on Druid (which is a huge dependency for them to take) or we would have to have some sort of compatibility layer between the two.

I do think that it is likely for Druid to have some classes that make it easier to use Memory for various operations, which would definitely live in Druid. The "read-only" version seems like something we should get the Memory library to support.

The non-native-endian problem basically means that our initial version of the integration will have to do non-native-endian reads (i.e. convert after reading non-native-endian value). Then, in a subsequent version, we will have to create a new serialization version that is only native endian.

The close/free stuff is already done.

The MagicAccessImpl thing looks cool. Is that basically a thing where we can put any arbitrary address and length in and it gives back a DirectByteBuffer that uses that? If so, that could be useful, yeah.

[leventov]

@cheddar I think sooner or later the proposed Memory should anyway become a part of Druid source, because it is a critical dependency and it's not good to depend on PR review/release cycle done by Yahoo exclusively, not Druid community. This is the same story as with java-util/bytebuffer-collections/extendedset and Metamarkets. Sooner, i. e. from the very beginning, is better.

Another solution is to move DataSketches lib to github.com/druid-io.

We could also add DataSketches's Memory <-> Druid's Memory conversion. Since Druid's Memory is supposed to be based on the DataSketches's one, it should be easy.

What do people from Imply (@gianm, @fjy) think on this question?

The MagicAccessImpl thing looks cool. Is that basically a thing where we can put any arbitrary address and length in and it gives back a DirectByteBuffer that uses that? If so, that could be useful, yeah.

I think it is needed for being able to make a refactoring gradually, not all-or-nothing. But it is super-ugly, likely won't work on any JDK except OpenJDK/Oracle JDK, likely won't work on Java 9 at all. So better it to be a temporary hack. If there are huge libraries which Druid depends on and they require ByteBuffer, it could also be a problem, because it means that the hack couldn't be removed.

[leerho]

@leventov @cheddar ,

+ Goal

• Eliminate current Endian craziness in Druid by standardizing on native-endian.

[cheddar]

Fwiw, I think it is better to try to do an all-or-nothing swap of the libraries and just make sure that we support the same persisted data. It's a much larger PR and will be a significant QA/code review risk, but this is also something that I think we either want to take whole-hog or not at all.

I propose we bring this up at the dev sync tomorrow, I'll make sure to mention it.

[leerho]

@leventov @cheddar Interesting discussion. I just have a few comments

• "Don't make a lot of Object copies when a Memory object has to be "sliced", ...". The MemoryRegion(R) is a view of its parent as "slice" is for ByteBuffer; no data is copied.

If the concern is the over-creation of these shell objects, with Memory you could create a read-only view of the parent once, and then just pass the offset and permitted length to the downstream users of some sub-chunk; no object creation at all. This is a little messy for the downstream users but safe. With BB this is not possible, because the position, limit, etc are always writable, which could screw up the parent.

• MagicAccessorImpl. I've been looking at this and it concerns me.

1. It brings in yet another hierarchy of dependencies (org.objectweb.asm), which could make maintenance & debugging rather challenging.
2. Creating a DirectByteBuffer requires calling the internal Bits.reserveMemory() and unreserveMemory(), which call the sun.misc.SharedSecrets() that keeps track of all created instances of DirectByteBuffer so that they can be properly freed. I don't see any mechanisms in the MagicAccessorImpl that does this. Of course, I may not fully grok how it works :).

• Using the JVM Cleaner. Although this might be possible, the Java gurus that I have spoken with, who work on the internals of the JVM, advise me to never use Cleaner as it can be quite dangerous (and hazardous to your mental health :) ). This and the above point are why I don't recommend trying to create a DirectByteBuffer from a Memory. Creating a HeapByteBuffer from Memory is trivial and can be done with the current code.

• Memory already has read-only implementations of NativeMemory and MemoryRegion.

• Memory has no dependencies outside Java, and you can think of it as a wrapper around Unsafe. This being the case, it would be possible to create classes in Memory that leverage the same UnsafeUtil and include position, limit, rewind, etc. But, is this really necessary? Being able to read old data should not be a problem, but using Memory is a different programming paradigm that new downstream applications would need to get used to.

• Memory was designed as a fast mechanism for random access of different primitive types sharing the same backing memory, similar to a C struct and union combined. ByteBuffer was designed for fast stream-based IO buffer operations, for which it does a splendid job. Reading a TCP/IP packet stream cannot be read with a random access paradigm so there is no choice. Reading data from RAM can be read either in a random access paradigm or sequentially, which depends on the application. But it seems to me to be overly restrictive for Druid to require that downstream processes must use a sequential paradigm in order to process their data out of RAM. Processing sketch data structures is a good example that requires random access, but I'm sure there will be many others.

Converting Druid away from ByteBuffers will be a lot of work no matter what path is chosen.

[gianm]

@leventov On the question of whether Memory should be part of Druid or not, my thought is that it is weird for Druid to depend on sketches-core, but if Memory stuff was broken out into its own library that both Druid and sketches-core used, then that could be fine. To me the decision of incorporating anything into the Druid tree or using it as a dependency should be made based on:

• Do we think we'll want to make changes to the library?
• Does the library have other users/contributors that we want to share development efforts with?

IMO, if the answer to those questions are "yes" and "no", respectively, for a given library then we should incorporate it into Druid. If the answers are anything else then it's better to keep it separate. I think for extendedset, bytebuffer-collections, and large portions of java-util, Druid was the only user so it made sense to merge them in. Memory has at least one other active user (datasketches) so to me that suggests including it as a dependency.

If that causes problems for Druid development we could always revisit.

[gianm]

@leerho Could you elaborate on what's wrong with Cleaner? It's been added recently to a few places in the Druid code base and if it's scary I'd like to understand why.

[gianm]

@cheddar

Fwiw, I think it is better to try to do an all-or-nothing swap of the libraries and just make sure that we support the same persisted data. It's a much larger PR and will be a significant QA/code review risk, but this is also something that I think we either want to take whole-hog or not at all.

Would you also propose changing factorizeBuffered to take Memory on the aggregators?

[cheddar]

@gianm yes, I would propose that the BufferedAggregators essentially become MemoryAggregators. A good amount of the speed gains that we can expect from the change will come from the intermediate processing layer, which means that the aggregators would have to change.

This will have wide-reaching affects on anyone that has done their own aggregators as extensions, but I think that it is the only way to actually make the change (and that's part of the risk of the change and why I think it will take a while to get it actually accepted)

[gianm]

Sounds like it's time to make the 0.11.0 milestone in github then!

[weijietong]

Does that mean a mmap call will return the Memory , aggregators will base on the Memory too ?