Upgrade Pinot service source version to Java 21

[gortiz]

Since 1.0.0, Pinot has dropped support for Java 8, and since then, the minimum source and runtime Java version is 11.
This document proposes increasing the minimum to Java 21 in the future.
This proposal doesn't affect the Pinot JDBC Driver or Pinot Connectors, which will still support Java 8. Pinot connectors are the drivers we provide for Flink and Spark. This means they are libraries/plugins that are designed to be deployed in Flink and Spark processes.

Current state

Pinot officially supports Java 11 and 21. At least, these are the versions we test.
Docker images use Java 11, 17 and 21.
The default docker image (the one tagged as latest) is Java 11.

A Java program compiled with Java 11 should be able to run on Java 11, 17, 21, 25, etc, without changes as
long as it only uses public APIs.
That is not the case for Pinot, which uses some internal APIs unavailable in Java 11.
These APIs are mainly used to get access to low-level features like network or memory management.
Specifically, we had some issues running in Java 21, given we were using some internal APIs through LArray to get
access to off-heap buffers larger than 2GBs.
These issues were fixed in #11656, and since Sept 2023
(#11672) all Pinot tests are executed in both Java 11 and 21.

Java versions

Java 8, 11, 17, and 21 are LTS versions, which officially means that Oracle supports them for a longer time than non-LTS versions.
In practice, other vendors like AdoptOpenJDK, Amazon, Azul, BellSoft, IBM, Red Hat, and others follow the same model
and support versions marked by Oracle as LTS for a longer time.

Java 11 is the first LTS version after Java 8, the minimum version for many projects.
The situation with Java 11 is becoming similar to that of Python 2.7, where many projects are stuck with it and it is hard to move forward.
This seems to be changing slowly.
Oracle and RedHat stopped supporting Java 11 in 2023 and 2024, respectively, and popular libraries/frameworks like
Spring Framework 6, Hibernate 6 or Micronaut 4 require Java 17.

Oracle support for Java 17 will end in 2026, and RedHat will support it until 2027.

Java 21 is the latest LTS version, and it is supported by Oracle until 2028 and by RedHat until 2029.

Java 25 is the next LTS release. It will be released in September 2025 and supported by Oracle until 2030 and by RedHat
until 2031.

Source vs runtime versions

The source version is the minimum Java version required to compile the code.
In Java this version is also the minimum runtime version.
Since Pinot 1.0.0, we are using Java 11 as the minimum source and runtime version, although the recommended runtime
version is Java 21, given it provides better performance and more runtime features like new GCs.

Cons of the update

Increasing the minimum Java version would have the inconvenience that users would need to install and use a newer Java
version.
This shouldn't be a problem given that Java 21 is already supported by all vendors and in fact, Java 11 is not supported
by most of them, but we know that some users move slowly to newer versions.
That is why, even though we can run Pinot in Java 21, we are still using 11 as the source version.

This should be completely transparent for users that run Pinot using Kubernetes or Docker, given the runtime Java version is included in the Docker images.

Pros of the update

The main reason to increase the minimum Java version is to be able to use new features and improvements in the Java
language and also runtime.

Although we can get some of the Java 21 performance improvements by running Pinot in Java 21, we can't use some of
them.
Things we cannot use go from nice to have programming features but also to important performance improvements, including:

• Code level improvements:
  • var keyword (https://openjdk.java.net/jeps/286)
  • instanceof pattern matching (https://openjdk.java.net/jeps/394),
  • switch expressions (https://openjdk.java.net/jeps/361)
  • sealed classes (https://openjdk.org/jeps/409)
  • record classes (https://openjdk.java.net/jeps/395)
• Performance improvements
  • Virtual threads (https://openjdk.org/jeps/444), which could be useful for MSE.
  • Foreign memory access (https://openjdk.org/jeps/442), which could be useful and faster than our off-heap memory access API.

Proposal

1. Open this issue to discuss how to proceed.
2. Get feedback from the community to try to find out if there are users who would be affected by this change.
3. If the community agrees, we will update the minimum Java version to Java 21. This change may be done for Pinot 1.4.0
   or for a future release.

Alternatives

Upgrade to Java 17 first

Although Java 17 is also LTS, it will be EOL for Oracle and RedHat in 2026 and 2027, respectively.
It also doesn't include any of the performance improvements listed above.
We also run tests in Java 21, so we are confident that Pinot works in Java 21.

We could consider upgrading to Java 17 as a first step to make the transition easier for our users,
but we would need to upgrade to Java 21 shortly.

Keep Java 11 as the minimum version and ignore new features

We could keep Java 11 as the minimum version and consider upgrading to Java 21 in a future release.
This would allow users to keep using Java 11 longer, but we would not be able to use new features and
some performance improvements in Java 21.

Keep Java 11 as the minimum version and use plugins for new features

We could keep Java 11 as the minimum version and use plugins to provide new features.
This means developers won't be able to use new features in the Java language, but we could provide some
performance improvements by using plugins.

The main disadvantage of this approach is that it would require more work from developers.
It is not easy to maintain a plugin that requires a Java version that is different from the main code.
This was, in fact, the main blocker of #12809 and why we decided to
stop working on it.
The corollary of this is that it is an incentive to create forks or separate projects that may not be open-source
to add support for these new features, which is not ideal.

[hpvd]

related to:
sum-up: things older java versions are blocking #14325

[yupeng9]

this could be a problem to customers who use Spark/Flink with Pinot.
Java 21 requires Spark4, and it's unclear what version of Flink that can support Java 21

[gortiz]

Datasketches 8.0.0 supports only Java 21 with --enable-preview https://github.com/apache/datasketches-java (https://lists.apache.org/thread/z7nr5wgkrrqnx61ytyqrblhnjpg5knly) and they have opened the discussion on how to help Java > 21 (https://lists.apache.org/thread/6ssxo917v7dxm8wfrb31o26qxbwhgskd)

I don't think we can accept --enable-preview by default in Pinot, so we may need to freeze datasketches until we move to Java > 21

[gortiz]

@yupeng9 do you mean pinot-connectors or ingestion plugins? pinot-connectors, like the pinot jdbc driver, should be independent of this change and still support Java 8/11. I've updated the issue description to clarify it.

[yupeng9]

the plugins like segment-uploader and segment-writer to push segments into pinot server.

[chenboat]

Moving to Java 21 will also enable new features like SIMD processing. @gortiz will this java21 upgrade go with 1.4 release?

[gortiz]

upgrade go with 1.4 release?

No, this is a long-term issue. I don't think we are going to upgrade the source version in the following months. It doesn't look major contributors are actually worried by this so it will take a while.

Moving to Java 21 will also enable new features like SIMD processing

Moving to source code to Java 21 is not a requirement to do so. First, remember that the JVM already tries to use SIMD operations when feasible (implicit vectorization). Second, explicit vectorization using Vector API is still in incubator (Java 25 will include the tenth incubator, see https://openjdk.org/jeps/508). Third, right now we can run Pinot in Java 21 (in fact it is the JVM we use by default). We can always create a plugin that uses Java 21 (or even newer) APIs. The main issue here is that we didn't create the machinery to include these plugins in the GHA pipelines of this repository.

Just to be clear, we will eventually increase the source version of the backend services (brokers, servers, controllers, minions, etc, without affecting drivers or external connectors) but it will take a while. The more contributors are interested in this issue, the better.

[chenboat]

@gortiz

Moving to source code to Java 21 is not a requirement to do so.

We use a lot of Lucene for our Pinot use cases. Lucene recently gets a lot of performance boost by utilizing Java 21 vector API. So moving to Java 21 source code could help us to get the best out of Lucnene

Third, right now we can run Pinot in Java 21 (in fact it is the JVM we use by default). We can always create a plugin that uses Java 21 (or even newer) APIs. The main issue here is that we didn't create the machinery to include these plugins in the GHA pipelines of this repository.

Can you elaborate how to do this? Plugin sounds a good option so that we do not need to fork.

[gortiz]

We use a lot of Lucene for our Pinot use cases. Lucene recently gets a lot of performance boost by utilizing Java 21 vector API. So moving to Java 21 source code could help us to get the best out of Lucnene

Luckily, we probably don't need to upgrade the source Java version to get these improvements. I'm not aware of how Lucene implemented the change, but probably they did it using multi-release jars, in which case we should get the performance boost for free just by changing the Java runtime version. So using Java 21 at runtime should be good enough.

Can you elaborate how to do this? Plugin sounds a good option so that we do not need to fork.

It is a trivial change. In the past, during #10528, we had a PR where we changed the Pinot buffers we used to use Foreign Memory API (see https://github.com/gortiz/pinot/blob/dc227182efaebff29e12320f5ba833ad269f6a54/pinot-plugins/pinot-foreign-memory/pom.xml) instead of larray (the older and deprecated code) or unsafe (the one we use right now). We decided to close that PR because we didn't have time to upgrade the GitHub Actions pipelines and Maven to compile only a sub-module using Java 21 and the rest using Java 11. This is totally possible to do, but given our pipelines take so long to run, it is an exhausting process we didn't have time to invest on.

[chenboat]

but probably they did it using multi-release jars

Thanks @gortiz for the suggestion and ideas. We already use Java 21 runtime in our production env. We can test and verify if they already supports Lucene's SIMD optimization. On the problem of multi-release jars, are they multiple jars? Then how to we know if Pinot uses the right jar?

[Jackie-Jiang]

@mcvsubbu @siddharthteotia @jadami10 What version of Java runtime are you running?