Address security vulnerabilities CVSS >= 7 by ccaominh · Pull Request #8980 · apache/druid

ccaominh · 2019-12-03T01:40:24Z

Description

Update dependencies to address security vulnerabilities with CVSS scores of 7 or higher. A new Travis CI job is added to prevent new high/critical security vulnerabilities from being added.

Updated dependencies:

api-util 1.0.0 -> 1.0.3
jackson 2.9.10 -> 2.10.1
kafka 2.1.0 -> 2.1.1
libthrift 0.10.0 -> 0.13.0
protobuf 3.2.0 -> 3.11.0

The following high/critical security vulnerabilities are currently suppressed (so that the new Travis CI job can be added now) and are left as future work:

hibernate-validator:5.2.5
jackson-mapper-asl:1.9.13
libthrift:0.6.1
netty:3.10.6
nimbus-jose-jwt:4.41.1

This PR has:

been self-reviewed.
added or updated version, license, or notice information in licenses.yaml
added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.

Update dependencies to address security vulnerabilities with CVSS scores of 7 or higher. A new Travis CI job is added to prevent new high/critical security vulnerabilities from being added. Updated dependencies: - api-util 1.0.0 -> 1.0.3 - jackson 2.9.10 -> 2.10.1 - kafka 2.1.0 -> 2.1.1 - libthrift 0.10.0 -> 0.13.0 - protobuf 3.2.0 -> 3.11.0 The following high/critical security vulnerabilities are currently suppressed (so that the new Travis CI job can be added now) and are left as future work to fix: - hibernate-validator:5.2.5 - jackson-mapper-asl:1.9.13 - libthrift:0.6.1 - netty:3.10.6 - nimbus-jose-jwt:4.41.1

ccaominh · 2019-12-04T03:45:18Z

Manually tested with https://druid.apache.org/docs/latest/tutorials/tutorial-batch-hadoop.html

clintropolis · 2019-12-04T07:19:55Z


 ---

+name: JavaBeans Activation Framework API JAR


Are these only run by CI? Unless these jars end up in the source binary packaging when building the distributions I don't believe they need to be included here.

They're part of the distribution now. Here's a diff of the distribution:

Only in before/extensions/druid-hdfs-storage: api-util-1.0.0-M20.jar Only in after/extensions/druid-hdfs-storage: api-util-1.0.3.jar Only in before/extensions/druid-kafka-extraction-namespace: kafka-clients-2.1.0.jar Only in after/extensions/druid-kafka-extraction-namespace: kafka-clients-2.1.1.jar Only in before/extensions/druid-kerberos: api-util-1.0.0-M20.jar Only in after/extensions/druid-kerberos: api-util-1.0.3.jar Only in after/extensions/druid-kinesis-indexing-service: jackson-annotations-2.10.1.jar Only in before/extensions/druid-kinesis-indexing-service: jackson-annotations-2.9.10.jar Only in after/extensions/druid-kinesis-indexing-service: jackson-core-2.10.1.jar Only in before/extensions/druid-kinesis-indexing-service: jackson-core-2.9.10.jar Only in after/extensions/druid-kinesis-indexing-service: jackson-databind-2.10.1.jar Only in before/extensions/druid-kinesis-indexing-service: jackson-databind-2.9.10.jar Only in before/extensions/druid-orc-extensions: protobuf-java-3.1.0.jar Only in after/extensions/druid-orc-extensions: protobuf-java-3.11.0.jar Only in after/extensions/druid-protobuf-extensions: error_prone_annotations-2.3.2.jar Only in before/extensions/druid-protobuf-extensions: gson-2.7.jar Only in after/extensions/druid-protobuf-extensions: gson-2.8.6.jar Only in after/extensions/druid-protobuf-extensions: protobuf-java-3.11.0.jar Only in before/extensions/druid-protobuf-extensions: protobuf-java-3.2.0.jar Only in after/extensions/druid-protobuf-extensions: protobuf-java-util-3.11.0.jar Only in before/extensions/druid-protobuf-extensions: protobuf-java-util-3.2.0.jar Only in after/lib: jackson-annotations-2.10.1.jar Only in before/lib: jackson-annotations-2.9.10.jar Only in after/lib: jackson-core-2.10.1.jar Only in before/lib: jackson-core-2.9.10.jar Only in after/lib: jackson-databind-2.10.1.jar Only in before/lib: jackson-databind-2.9.10.jar Only in after/lib: jackson-dataformat-cbor-2.10.1.jar Only in before/lib: jackson-dataformat-cbor-2.9.10.jar Only in after/lib: jackson-dataformat-smile-2.10.1.jar Only in before/lib: jackson-dataformat-smile-2.9.10.jar Only in after/lib: jackson-datatype-guava-2.10.1.jar Only in before/lib: jackson-datatype-guava-2.9.10.jar Only in after/lib: jackson-datatype-joda-2.10.1.jar Only in before/lib: jackson-datatype-joda-2.9.10.jar Only in after/lib: jackson-jaxrs-base-2.10.1.jar Only in before/lib: jackson-jaxrs-base-2.9.10.jar Only in after/lib: jackson-jaxrs-json-provider-2.10.1.jar Only in before/lib: jackson-jaxrs-json-provider-2.9.10.jar Only in after/lib: jackson-jaxrs-smile-provider-2.10.1.jar Only in before/lib: jackson-jaxrs-smile-provider-2.9.10.jar Only in after/lib: jackson-module-guice-2.10.1.jar Only in before/lib: jackson-module-guice-2.9.10.jar Only in after/lib: jackson-module-jaxb-annotations-2.10.1.jar Only in before/lib: jackson-module-jaxb-annotations-2.9.10.jar Only in after/lib: jakarta.activation-api-1.2.1.jar Only in after/lib: jakarta.xml.bind-api-2.3.2.jar Only in before/lib: protobuf-java-3.1.0.jar Only in after/lib: protobuf-java-3.11.0.jar Only in after/licenses/bin: jakarta.activation-api.EDL1

clintropolis · 2019-12-04T07:20:22Z

@@ -0,0 +1,13 @@
+Eclipse Distribution License - v 1.0


Likewise these might not be needed depending on the answer to whether or not this stuff is only used by CI

clintropolis · 2019-12-04T07:22:40Z

  private JsonParser jp;
  private ObjectCodec objectCodec;
-  private final TypeReference typeRef;
+  private final TypeReference<T> typeRef;


Are these changes required for something or just opportunistic?

They are required after upgrading Jackson to 2.10

ccaominh · 2019-12-04T18:22:14Z

  private JsonParser jp;
  private ObjectCodec objectCodec;
-  private final TypeReference typeRef;
+  private final TypeReference<T> typeRef;


They are required after upgrading Jackson to 2.10

ccaominh · 2019-12-04T18:23:25Z

  public List<String> getMetadataSegments(final String dataSource)
  {
-    ArrayList<String> segments;
+    List<String> segments;


This was required after upgrading Jackson to 2.10 (since TypeReference<List<String>> is used below)

ccaominh · 2019-12-04T18:24:39Z


 ---

+name: JavaBeans Activation Framework API JAR


They're part of the distribution now. Here's a diff of the distribution:

Only in before/extensions/druid-hdfs-storage: api-util-1.0.0-M20.jar Only in after/extensions/druid-hdfs-storage: api-util-1.0.3.jar Only in before/extensions/druid-kafka-extraction-namespace: kafka-clients-2.1.0.jar Only in after/extensions/druid-kafka-extraction-namespace: kafka-clients-2.1.1.jar Only in before/extensions/druid-kerberos: api-util-1.0.0-M20.jar Only in after/extensions/druid-kerberos: api-util-1.0.3.jar Only in after/extensions/druid-kinesis-indexing-service: jackson-annotations-2.10.1.jar Only in before/extensions/druid-kinesis-indexing-service: jackson-annotations-2.9.10.jar Only in after/extensions/druid-kinesis-indexing-service: jackson-core-2.10.1.jar Only in before/extensions/druid-kinesis-indexing-service: jackson-core-2.9.10.jar Only in after/extensions/druid-kinesis-indexing-service: jackson-databind-2.10.1.jar Only in before/extensions/druid-kinesis-indexing-service: jackson-databind-2.9.10.jar Only in before/extensions/druid-orc-extensions: protobuf-java-3.1.0.jar Only in after/extensions/druid-orc-extensions: protobuf-java-3.11.0.jar Only in after/extensions/druid-protobuf-extensions: error_prone_annotations-2.3.2.jar Only in before/extensions/druid-protobuf-extensions: gson-2.7.jar Only in after/extensions/druid-protobuf-extensions: gson-2.8.6.jar Only in after/extensions/druid-protobuf-extensions: protobuf-java-3.11.0.jar Only in before/extensions/druid-protobuf-extensions: protobuf-java-3.2.0.jar Only in after/extensions/druid-protobuf-extensions: protobuf-java-util-3.11.0.jar Only in before/extensions/druid-protobuf-extensions: protobuf-java-util-3.2.0.jar Only in after/lib: jackson-annotations-2.10.1.jar Only in before/lib: jackson-annotations-2.9.10.jar Only in after/lib: jackson-core-2.10.1.jar Only in before/lib: jackson-core-2.9.10.jar Only in after/lib: jackson-databind-2.10.1.jar Only in before/lib: jackson-databind-2.9.10.jar Only in after/lib: jackson-dataformat-cbor-2.10.1.jar Only in before/lib: jackson-dataformat-cbor-2.9.10.jar Only in after/lib: jackson-dataformat-smile-2.10.1.jar Only in before/lib: jackson-dataformat-smile-2.9.10.jar Only in after/lib: jackson-datatype-guava-2.10.1.jar Only in before/lib: jackson-datatype-guava-2.9.10.jar Only in after/lib: jackson-datatype-joda-2.10.1.jar Only in before/lib: jackson-datatype-joda-2.9.10.jar Only in after/lib: jackson-jaxrs-base-2.10.1.jar Only in before/lib: jackson-jaxrs-base-2.9.10.jar Only in after/lib: jackson-jaxrs-json-provider-2.10.1.jar Only in before/lib: jackson-jaxrs-json-provider-2.9.10.jar Only in after/lib: jackson-jaxrs-smile-provider-2.10.1.jar Only in before/lib: jackson-jaxrs-smile-provider-2.9.10.jar Only in after/lib: jackson-module-guice-2.10.1.jar Only in before/lib: jackson-module-guice-2.9.10.jar Only in after/lib: jackson-module-jaxb-annotations-2.10.1.jar Only in before/lib: jackson-module-jaxb-annotations-2.9.10.jar Only in after/lib: jakarta.activation-api-1.2.1.jar Only in after/lib: jakarta.xml.bind-api-2.3.2.jar Only in before/lib: protobuf-java-3.1.0.jar Only in after/lib: protobuf-java-3.11.0.jar Only in after/licenses/bin: jakarta.activation-api.EDL1

ccaominh · 2019-12-04T18:25:43Z

+license_name: Eclipse Distribution License 1.0
+version: 2.3.2
+copyright: Oracle and/or its affiliates.
+license_file_path: licenses/bin/jakarta.activation-api.EDL1


Since the EDL1 license is used for both jakarta.activation-api and jakarta.xml.bind-api, I'll rename the license file to "jakarta.EDL1"

ccaominh force-pushed the cve-check branch 8 times, most recently from 2191a73 to f2ac3a5 Compare December 4, 2019 00:08

ccaominh force-pushed the cve-check branch from f2ac3a5 to 2e129d7 Compare December 4, 2019 01:55

ccaominh marked this pull request as ready for review December 4, 2019 03:45

clintropolis reviewed Dec 4, 2019

View reviewed changes

Rename EDL1 license file

0c3809e

ccaominh commented Dec 4, 2019

View reviewed changes

Fix inspection errors

de9a4cc

clintropolis added Area - Dependencies Security labels Dec 4, 2019

clintropolis approved these changes Dec 5, 2019

View reviewed changes

clintropolis merged commit af74aca into apache:master Dec 5, 2019

ccaominh deleted the cve-check branch December 6, 2019 02:09

ccaominh mentioned this pull request Dec 13, 2019

[WIP] Address security vulnerabilities #8664

Closed

jon-wei added this to the 0.17.0 milestone Dec 17, 2019

jon-wei added the Release Notes label Dec 19, 2019

jon-wei mentioned this pull request Dec 28, 2019

0.17.0 release notes #9066

Closed

ccaominh mentioned this pull request Mar 10, 2020

Druid Vulnerability Analysis #8432

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Address security vulnerabilities CVSS >= 7#8980

Address security vulnerabilities CVSS >= 7#8980
clintropolis merged 3 commits intoapache:masterfrom
ccaominh:cve-check

ccaominh commented Dec 3, 2019 •

edited

Loading

Uh oh!

ccaominh commented Dec 4, 2019

Uh oh!

clintropolis Dec 4, 2019

Uh oh!

ccaominh Dec 4, 2019

Uh oh!

clintropolis Dec 4, 2019

Uh oh!

clintropolis Dec 4, 2019

Uh oh!

ccaominh Dec 4, 2019

Uh oh!

ccaominh Dec 4, 2019

Uh oh!

ccaominh Dec 4, 2019

Uh oh!

ccaominh Dec 4, 2019

Uh oh!

ccaominh Dec 4, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

ccaominh commented Dec 3, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Uh oh!

ccaominh commented Dec 4, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ccaominh commented Dec 3, 2019 •

edited

Loading