[Java] Finish support for 64 bit int allocations

[asfimport]

1.  Add an allocator capable of allocating larger then 2GB of data.

2.  Do end-to-end round trip trip on a larger vector/record batch size.

Reporter: Micah Kornfield / @emkornfield
Assignee: Liya Fan / @liyafan82

Related issues:

• [Java] Support large buffer for file/stream IPC (blocks)

PRs and other links:

• GitHub Pull Request #6323

_{Note: This issue was originally created as ARROW-7610. Please see the migration documentation for further details.}

[asfimport]

Liya Fan / @liyafan82:
may i try to resolve this issue？

[asfimport]

Micah Kornfield / @emkornfield:
[~fan_li_ya] that would be great, I started a thread on the mailing list about possible options for an allocator.

[asfimport]

Liya Fan / @liyafan82:
i see. thanks.
this seems interesting, and i am working on another issue related to allocator

[asfimport]

Jacques Nadeau / @jacques-n:
Per comments on the mailing list,

I think we should use the existing netty facades for this.

PlatformDependent.allocateMemory(long)
PlatformDependent.freeMemory(long)

These delegate to unsafe which also ensures that -XX:MaxDirectMemorySize is respected.

I don't think we need a new allocator. We can simply check the size and use this path if beyond an int in size within the existing allocator.

[asfimport]

Bryan Cutler / @BryanCutler:
Issue resolved by pull request 6323
#6323