Jackson 2.4.6 incompatible with HDP 2.4

[waltonseymour]

When trying to submit a index_hadoop job to a HDP 2.4 cluster, I get the following error:

Error: class com.fasterxml.jackson.datatype.guava.deser.HostAndPortDeserializer overrides final method deserialize

From looking around, it seems to be a jackson version incomparability issue referenced by #1356.
I've tried building from source with an older version of jackson (2.3.2), but this fails to compile.
Is there a decent workaround for this issue?

[erikdubbelboer]

The way we worked around this is to replace the HDP jackson jars with the ones from Druid so they both use the same version. This worked for us with Hadoop 2.7.2.

[ashwing]

@waltonseymour You may have to look out for two things.

1. If you are composing a fat lib jar to be submitted to druid, make sure you clean the old jars and libs in druid'd hdfs classpath, as the new jars will not overwrite the old jars having the same filename.
2. And as Erik suggested, you have to make sure the version of jackson jars in druid's hdfs classpath or hadoop environment, matches with that of the jars come with druid core. You can tweak your build script by eliminating druid's jackson jars with Exclusion Rules and explicitly adding the required jackson jars.

[waltonseymour]

I went down the fat lib jar route but ran into
io.druid.initialization - Could not find artifact org.apache.hadoop:hadoop-client:jar:2.7.1.
I cleaned out the jars from HDFS and tried a number of different things with no luck.

Do I need to recompile druid against Hadoop 2.7.1 also?

[waltonseymour]

As a note, I was able to get druid working with HDP 2.4 by reverting #1989 and recompiling with an older jackson version.

[slaunay]

@waltonseymour I have been using a simpler workaround described in this PR: #2815.

You can try it with the following configuration in your task if you want to continue using the Vanilla version of Druid:

    "tuningConfig" : {
      ...
      "jobProperties" : {
        "mapreduce.job.user.classpath.first": "true"
      },
      ...
    }

[santhoshtangudu]

We are using Hortonwork Hadoop distribution in production. While running this job, even I set the flag, it didn't work well. Still I am getting this error. My cluster is using Jackson 2.2.3 version and Druid is using 2.4.6 version. Can someone suggest me to solve the problem without replacing Cluster(Shared Cluster) Jackson Libraries ?

[fjy]

@santhoshtangudu have you looked at the steps described in http://druid.io/docs/0.9.2-rc1/operations/other-hadoop.html? It should help with your issue.

[santhoshtangudu]

We were passing multiple versions of Jackson libraries. I stopped passing multiple versions and then the problem is fixed.

@fjy, thanks for quick help.

[MTShannon]

@santhoshtangudu How did you stop passing multiple versions?