Require Java 8 runtime

[gianm]

Some reasons to require a Java 8 runtime: #3745, #3595, #3710.

Related mailing list discussions:

• https://groups.google.com/d/topic/druid-development/6I9uhO3ZIeQ/discussion
• https://groups.google.com/d/topic/druid-user/aTGQlnF1KLk/discussion

Things we should verify first:

• Embedding inside Spark still works, @drcrallen to verify
• Embedding inside Tranquility still works, @gianm to verify

It's also possible to use Java 8 source and bytecode but I think that's a separate discussion, that IMO should come with some patches that use the new Java 8 features to good effect.

[drcrallen]

See #3328 for byte-code java 8 related stuff.

[gianm]

Just to be clear, I'm not intending to drive a change to Java 8 source and bytecode. Just requiring Java 8 JVM at runtime so we can use dependencies that do use Java 8 bytecode (like newer versions of Jetty and DataSketches).

I would be OK with changing to Java 8 source and bytecode as well if the community thinks it's worth it, but that is not what I meant to propose.

[gianm]

IMO a proposal to switch to Java 8 source should involve some patches that use the new Java 8 features to good effect.

[gianm]

I read through the druid-user poll and the two common themes for the "keep java 7 support" vote seem to be:

• Concerns about Scala compatibility (we already captured that in this ticket)
• Concerns about Hadoop, since a few people are stuck on Java 7 based Hadoop clusters that are seemingly out of their control to upgrade. I can see this being the case for centrally managed Hadoop clusters, especially if the team behind it doesn't upgrade very often.

I'm not sure if this is feasible, but we might be able to keep the Hadoop indexer working on Java 7 runtimes even as the services move to requiring Java 8. The Hadoop indexer shouldn't need Jetty and if we're careful about what other Java 8 dependencies we add then we could possibly make that work.

[drcrallen]

@gianm / @dclim do you guys have a test environment setup that could capture a java7 hadoop environment?

[tjgiuli]

From Cloudera's documentation (https://www.cloudera.com/documentation/enterprise/5-3-x/topics/cdh_cm_upgrading_to_jdk8.html): "Cloudera does not support a rolling upgrade to JDK 1.8. You must shut down the entire cluster." Pretty unfortunate.

[gianm]

@drcrallen our "hadoop gauntlet" tests for Apache Hadoop distros run in jdk-7u71. I'm not sure about our tests for other distros, but if the Apache ones work then that's a good sign.