Skip to content

exclude kerberos extension dependencies that are already included in druid core libraries#8310

Merged
jon-wei merged 5 commits intoapache:masterfrom
clintropolis:chillax-kerberos-dependencies
Aug 16, 2019
Merged

exclude kerberos extension dependencies that are already included in druid core libraries#8310
jon-wei merged 5 commits intoapache:masterfrom
clintropolis:chillax-kerberos-dependencies

Conversation

@clintropolis
Copy link
Copy Markdown
Member

@clintropolis clintropolis commented Aug 15, 2019

Description

While working on NOTICE stuff for 0.16.0, noticed kerberos extension had a lot of dependencies that were already included in lib/ folder of package, so I have excluded them from the kerberos extension pom.

before:
Screen Shot 2019-08-14 at 8 55 21 PM

$ ls -1
apacheds-i18n-2.0.0-M15.jar
apacheds-kerberos-codec-2.0.0-M15.jar
api-asn1-api-1.0.0-M20.jar
api-util-1.0.0-M20.jar
commons-beanutils-1.7.0.jar
commons-beanutils-core-1.8.0.jar
commons-collections-3.2.2.jar
commons-compress-1.18.jar
commons-configuration-1.6.jar
commons-digester-1.8.jar
commons-net-3.6.jar
druid-kerberos-0.16.0-incubating-SNAPSHOT.jar
gson-2.2.4.jar
hadoop-auth-2.8.3.jar
hadoop-common-2.8.3.jar
htrace-core4-4.0.1-incubating.jar
jackson-jaxrs-1.9.2.jar
jackson-xc-1.9.2.jar
java-xmlbuilder-0.4.jar
javax.activation-api-1.2.0.jar
javax.servlet-api-3.1.0.jar
jaxb-api-2.3.1.jar
jaxb-impl-2.2.3-1.jar
jcip-annotations-1.0.jar
jersey-json-1.19.3.jar
jersey-server-1.19.3.jar
jets3t-0.9.0.jar
jettison-1.1.jar
jetty-6.1.26.jar
jetty-client-9.4.10.v20180503.jar
jetty-continuation-9.4.10.v20180503.jar
jetty-http-9.4.10.v20180503.jar
jetty-io-9.4.10.v20180503.jar
jetty-proxy-9.4.10.v20180503.jar
jetty-security-9.4.10.v20180503.jar
jetty-server-9.4.10.v20180503.jar
jetty-servlet-9.4.10.v20180503.jar
jetty-servlets-9.4.10.v20180503.jar
jetty-sslengine-6.1.26.jar
jetty-util-9.4.10.v20180503.jar
jsch-0.1.54.jar
json-smart-1.1.1.jar
jsp-api-2.1.jar
nimbus-jose-jwt-3.9.jar
servlet-api-2.5.jar
xmlenc-0.52.jar

after:
Screen Shot 2019-08-14 at 8 22 41 PM

$ ls -1
apacheds-i18n-2.0.0-M15.jar
apacheds-kerberos-codec-2.0.0-M15.jar
api-asn1-api-1.0.0-M20.jar
api-util-1.0.0-M20.jar
commons-beanutils-core-1.8.0.jar
commons-configuration-1.6.jar
druid-kerberos-0.16.0-incubating-SNAPSHOT.jar
gson-2.2.4.jar
hadoop-auth-2.8.3.jar
hadoop-common-2.8.3.jar
htrace-core4-4.0.1-incubating.jar
jackson-jaxrs-1.9.2.jar
jackson-xc-1.9.2.jar
java-xmlbuilder-0.4.jar
javax.activation-api-1.2.0.jar
jaxb-api-2.3.1.jar
jaxb-impl-2.2.3-1.jar
jcip-annotations-1.0.jar
jersey-json-1.19.3.jar
jersey-server-1.19.3.jar
jets3t-0.9.0.jar
jettison-1.1.jar
jsch-0.1.54.jar
json-smart-1.1.1.jar
jsp-api-2.1.jar
nimbus-jose-jwt-3.9.jar
xmlenc-0.52.jar

Looking at the actual classes the production dependencies use, this change appears safe to me, and in fact we could probably remove even more, but I haven't tested for reals so staying conservative-ish...

Screen Shot 2019-08-14 at 10 32 21 PM

Copy link
Copy Markdown
Member

@asdf2014 asdf2014 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍 BTW, there are still conflicts of dependence. Can we solve them too?

@clintropolis
Copy link
Copy Markdown
Member Author

Oops, looks like json-smart can be excluded, others look like issues not specific to this extension, hopefully #8230 can resolve some of those

@clintropolis
Copy link
Copy Markdown
Member Author

Decided to just go ahead and try removing most of the ones intellij indicated we shouldn't need:

Screen Shot 2019-08-15 at 1 38 28 AM

@clintropolis clintropolis added this to the 0.16.0 milestone Aug 15, 2019
@clintropolis
Copy link
Copy Markdown
Member Author

Running some tests to make sure everything is still good after these changes

@clintropolis
Copy link
Copy Markdown
Member Author

Tested in a kerberized docker environment, http requests were functioning but, ran into an issue with it complaining about missing commons-configuration initially when trying to run an indexing task. After adding back commons-configuration indexing completed successfully so I believe this should be in good shape now.

@clintropolis clintropolis removed the WIP label Aug 16, 2019
@clintropolis
Copy link
Copy Markdown
Member Author

final set of dependencies after this PR:
Screen Shot 2019-08-16 at 3 39 25 PM

@jon-wei jon-wei merged commit a34cfd3 into apache:master Aug 16, 2019
@clintropolis clintropolis deleted the chillax-kerberos-dependencies branch August 16, 2019 22:46
jihoonson pushed a commit to implydata/druid-public that referenced this pull request Aug 23, 2019
…druid core libraries (apache#8310)

* exclude kerberos extension dependencies that are already included in druid libs

* missing net

* exclude json-smart

* eh might as well go aggro and remove all the ones it looks like we do not need

* guess we actually need this one
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants