[Java] Arrow's use of Netty library causes UnsupportedOperationException with JDK16

[asfimport]

The Arrow vector library depends on netty, which throws the exception java.lang.UnsupportedOperationException: sun.misc.Unsafe or java.nio.DirectByteBuffer.(long, int) not available. In the documentation, it's stated that setting "-Dio.netty.tryReflectionSetAccessible=true" fixes the issue. This is true with all java versions except with usage of the latest JDK16. The exception gets thrown regardless of this system property setting when using JDK16.

 

Is there a fix for this?

Thanks!

Reporter: Megan Knister
Assignee: David Dali Susanibar Arce / @davisusanibar

Related issues:

• [Java][CI]: Add support for Java 17 CI process (requires)
• [Java] Support Java 17 (is required by)

_{Note: This issue was originally created as ARROW-12747. Please see the migration documentation for further details.}

[asfimport]

Mark Diskin:
The next Java LTS (17) is now released when will this be supported? This will be the baseline for many frameworks to adopt the jakarta open source releases  e.g. https://spring.io/blog/2021/09/02/a-java-17-and-jakarta-ee-9-baseline-for-spring-framework-6

https://www.oracle.com/news/announcement/oracle-releases-java-17-2021-09-14/

[asfimport]

Megan Knister:
Hi, this is still in issue in the most recent Arrow version (5.0.0). When will this be addressed?

 

Thanks,

Meg

[asfimport]

Reid McKenzie:
The underlying problem is in the depended version of Netty. 4.1.52+ has JDK17 fixes. Excluding the dependencies arrow uses and using netty-{buffer,common} 4.1.57.Final appears to resolve this issue.

[asfimport]

Marius Oancea:
I'm using Arrow 0.16.0 with netty 4.1.72.Final and the problem still exists (JDK17):

Error occured while deserializing arrow data: sun.misc.Unsafe or java.nio.DirectByteBuffer.(long, int) not available 

 

{}Update{}: I got it to work using java option  --add-opens java.base/java.nio=ALL-UNNAMED 

[asfimport]

David Dali Susanibar Arce / @davisusanibar:
Hi [~meg.knister@snowflake.com]  please could you try to test changes on this PR on your JSE16 environment, just tested on my side and it is working. Please let me know if you have some errors.

[asfimport]

David Dali Susanibar Arce / @davisusanibar:
Hi [~meg.knister@snowflake.com] this PR #12941 was merged and just tested master branch with java version "16.0.2" and if finished without errors, I am going to close this issue and please let us know if you find some problems.