Skip to content

Conversation

@adoroszlai
Copy link
Contributor

What changes were proposed in this pull request?

Several tests fail when run with Java 17 due to:

java.lang.ExceptionInInitializerError
	at org.apache.hadoop.ozone.common.TestChecksumImplsComputeSameValues.testCRC32CImplsMatch(TestChecksumImplsComputeSameValues.java:66)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make field boolean java.nio.ByteBuffer.isReadOnly accessible: module java.base does not "opens java.nio" to unnamed module @75eeccf5
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
	at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:178)
	at java.base/java.lang.reflect.Field.setAccessible(Field.java:172)
	at org.apache.hadoop.ozone.common.ChecksumByteBufferImpl.<clinit>(ChecksumByteBufferImpl.java:44)
	... 4 more

ozone command does open some packages for ChecksumByteBufferImpl to work (see #3759). This change applies the same logic (extracted to ozone-functions.sh) in junit.sh to Surefire argLine.

https://issues.apache.org/jira/browse/HDDS-10123

How was this patch tested?

Ran tests that failed previously with Java 17.

Example:

$ JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64 ./hadoop-ozone/dev-support/checks/junit.sh -am -pl :hdds-common -Dtest='TestChecksumImplsComputeSameValues'
...
Java version: 17.0.9
...
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.206 s -- in org.apache.hadoop.ozone.common.TestChecksumImplsComputeSameValues

CI (also verifies that refactoring bin/ozone did not cause regression):
https://github.com/adoroszlai/ozone/actions/runs/7505840912

@adoroszlai adoroszlai self-assigned this Jan 12, 2024
OZONE_MODULE_ACCESS_ARGS="${OZONE_MODULE_ACCESS_ARGS} --add-opens java.management/com.sun.jmx.mbeanserver=ALL-UNNAMED"
OZONE_MODULE_ACCESS_ARGS="${OZONE_MODULE_ACCESS_ARGS} --add-exports java.management/com.sun.jmx.mbeanserver=ALL-UNNAMED"
fi
if [[ "${JAVA_MAJOR_VERSION}" -ge 9 ]]; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it intended for this branch to run for Java 17 as well as Java 9?

I think it will result in two --add-opens switches in the args:

--add-opens java.base/java.lang=ALL-UNNAMED"
--add-opens java.base/java.nio=ALL-UNNAMED"

I think this is OK, but just checking.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. The first part (specific to Java 17) was added for HDDS-8533. Ozone Cluster components (SCM, OM, DNs) fails to start with JDK 17. The second part got added later for HDDS-7228. ChecksumByteBufferImpl.update() is expensive. I just moved these from ozone to ozone-functions.sh.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool - sounds good.

Copy link
Contributor

@sodonnel sodonnel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - I just had one clarification on a part of the change.

@adoroszlai adoroszlai merged commit 7b90ca7 into apache:master Jan 15, 2024
@adoroszlai adoroszlai deleted the HDDS-10123 branch January 15, 2024 17:00
@adoroszlai
Copy link
Contributor Author

Thanks @sodonnel for the review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants