Bump Apache Avro to 1.9.0#7772
Conversation
|
It seems to clash with Apache Pig which still uses Avro 1.7.4. |
|
Would it be possible to update Pig too? |
|
Good point. I'm looking into this. Pig is very old and even the latest version uses Avro 1.7.5. |
Apache Avro 1.9.0 brings a lot of new features: * Deprecate Joda-Time in favor of Java8 JSR310 and setting it as default * Remove support for Hadoop 1.x * Move from Jackson 1.x to 2.9 * Add ZStandard Codec * Lots of updates on the dependencies to fix CVE's * Remove Jackson classes from public API * Apache Avro is built by default with Java 8 * Apache Avro is compiled and tested with Java 11 to guarantee compatibility * Apache Avro MapReduce is compiled and tested with Hadoop 3 * Apache Avro is now leaner, multiple dependencies were removed: guava, paranamer, commons-codec, and commons-logging * Introduce JMH Performance Testing Framework * Add Snappy support for C++ DataFile * and many, many more!
|
Rebased onto master |
| <exclusion> | ||
| <groupId>org.codehaus.jackson</groupId> | ||
| <artifactId>jackson-mapper-asl</artifactId> | ||
| </exclusion> |
There was a problem hiding this comment.
I noticed that instead of these, avro-mapred version 1.9.0 includes com.fasterxml.jackson.core version 2.9.8, will that cause any problems?
There was a problem hiding this comment.
Yes, it moved from Jackson 1.x org.codehaus.jackson to 2.x com.fasterxml.jackson. If this is a problem, it depends. I think for Druid itself it will pan out, but in combination with different Hadoop versions, as always, there might be issues.
I also have a PR to bump Druid to the same version: #7701. As you can see there are no breaking API changes, so I don't expect a lot of issues.
There was a problem hiding this comment.
I'm fine with excluding the com.fasterxml.jackson from Apache Avro if you prefer that.
There was a problem hiding this comment.
Excluding here I think would be best if everything still works.
There was a problem hiding this comment.
Excluding is the safest path, I've added the exclusions 👍
|
Since the version of Avro was changed, the LICENSE.BINARY file should have updated in this PR accordingly. This will be automatically checked after #7998. |
* Bump Apache Avro to 1.9.0 Apache Avro 1.9.0 brings a lot of new features: * Deprecate Joda-Time in favor of Java8 JSR310 and setting it as default * Remove support for Hadoop 1.x * Move from Jackson 1.x to 2.9 * Add ZStandard Codec * Lots of updates on the dependencies to fix CVE's * Remove Jackson classes from public API * Apache Avro is built by default with Java 8 * Apache Avro is compiled and tested with Java 11 to guarantee compatibility * Apache Avro MapReduce is compiled and tested with Hadoop 3 * Apache Avro is now leaner, multiple dependencies were removed: guava, paranamer, commons-codec, and commons-logging * Introduce JMH Performance Testing Framework * Add Snappy support for C++ DataFile * and many, many more! * Add exclusions for Jackson
4 participants
Apache Avro 1.9.0 brings a lot of new features: