Gigantic jar when using Gradle 8.x

[jimshowalter]

Please check the User Guide before submitting "how do I do 'x'?" questions!

Shadow Version

8.1.1

Gradle Version

8.6

Expected Behavior

Should only need to shadow bouncycastle dependencies, and resulting jar should be small, with just bouncycastle relocated.

Actual Behavior

Without adding a number of other dependencies, the resulting jar was not usable by our code, which failed at runtime with missing-class exceptions.

See comment in shadow-bad's dependency.gradle that starts with "// If this dependencies{} block is commented out".

shadow-good.zip

shadow-bad.zip

[jimshowalter]

When we try to exclude, it has no effect:

shadowJar {
    dependencies {
        include(dependency("org.bouncycastle:bcpg-jdk18on:1.77"))
        include(dependency("org.bouncycastle:bcpkix-jdk18on:1.77"))
        include(dependency("org.bouncycastle:bcprov-jdk18on:1.77"))
        include(dependency("org.bouncycastle:bcutil-jdk18on:1.77"))
        exclude(dependency('ch.qos.logback:logback-classic:1.4.14'))
        exclude(dependency('ch.qos.logback:logback-core:1.4.14'))
        exclude(dependency('com.google.code.findbugs:jsr305:3.0.2'))
        exclude(dependency('com.google.errorprone:error_prone_annotations:2.23.0'))
        exclude(dependency('com.google.guava:failureaccess:1.0.2'))
        exclude(dependency('com.google.guava:guava:33.0.0-jre'))
        exclude(dependency('com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava'))
        exclude(dependency('com.google.j2objc:j2objc-annotations:2.8'))
        exclude(dependency('commons-codec:commons-codec:1.11'))
        exclude(dependency('commons-codec:commons-codec:1.16.1'))
        exclude(dependency('commons-io:commons-io:2.15.1'))
        exclude(dependency('io.netty:netty-buffer:4.1.104.Final'))
        exclude(dependency('io.netty:netty-common:4.1.104.Final'))
        exclude(dependency('io.netty:netty-resolver:4.1.104.Final'))
        exclude(dependency('io.netty:netty-transport-classes-epoll:4.1.104.Final'))
        exclude(dependency('io.netty:netty-transport-native-epoll:4.1.104.Final'))
        exclude(dependency('io.netty:netty-transport-native-unix-common:4.1.104.Final'))
        exclude(dependency('io.netty:netty-transport:4.1.104.Final'))
        exclude(dependency('net.bytebuddy:byte-buddy-agent:1.14.11'))
        exclude(dependency('net.bytebuddy:byte-buddy:1.14.11'))
        exclude(dependency('org.apache.commons:commons-lang3:3.14.0'))
        exclude(dependency('org.apache.httpcomponents:httpclient:4.5.14'))
        exclude(dependency('org.apache.httpcomponents:httpcore:4.4.16'))
        exclude(dependency('org.apache.logging.log4j:log4j-api:2.20.0'))
        exclude(dependency('org.apache.logging.log4j:log4j-to-slf4j:2.20.0'))
        exclude(dependency('org.checkerframework:checker-qual:3.41.0'))
        exclude(dependency('org.jacoco:org.jacoco.agent:0.8.10'))
        exclude(dependency('org.jacoco:org.jacoco.ant:0.8.10'))
        exclude(dependency('org.junit.jupiter:junit-jupiter-api:5.10.1'))
        exclude(dependency('org.junit.jupiter:junit-jupiter-api:5.10.2'))
        exclude(dependency('org.junit.jupiter:junit-jupiter-engine:5.10.2'))
        exclude(dependency('org.junit.platform:junit-platform-commons:1.10.2'))
        exclude(dependency('org.junit.platform:junit-platform-engine:1.10.2'))
        exclude(dependency('org.junit:junit-bom:5.10.2'))
        exclude(dependency('org.mockito:mockito-core:5.10.0'))
        exclude(dependency('org.mockito:mockito-junit-jupiter:5.10.0'))
        exclude(dependency('org.mongodb:bson-record-codec:4.11.1'))
        exclude(dependency('org.mongodb:bson:4.11.1'))
        exclude(dependency('org.mongodb:mongodb-driver-core:4.11.1'))
        exclude(dependency('org.mongodb:mongodb-driver-sync:4.11.1'))
        exclude(dependency('org.objenesis:objenesis:3.3'))
        exclude(dependency('org.opentest4j:opentest4j:1.3.0'))
        exclude(dependency('org.projectlombok:lombok:1.18.30'))
        exclude(dependency('org.slf4j:jcl-over-slf4j:1.7.36'))
        exclude(dependency('org.slf4j:jul-to-slf4j:1.7.36'))
        exclude(dependency('org.slf4j:log4j-over-slf4j:1.7.36'))
        exclude(dependency('org.slf4j:slf4j-api:1.7.25'))
        exclude(dependency('org.slf4j:slf4j-api:1.7.36'))
        exclude(dependency('org.slf4j:slf4j-api:2.0.7'))
        exclude(dependency('org.slf4s:slf4s-api_2.10:1.7.25'))
    }
    zip64 true
    // Removes "-all" from the end of the artifact name, to match what is expected.
    archiveClassifier = null
    // Change the classpath of bouncy castle to avoid conflicts with client's versions.
    // https://confluence.inside-box.net/display/ETO/The+FIPS+Problem+And+Possible+Solutions
    relocate 'org.bouncycastle', 'shadow.bouncycastle'
}

results in the contents shown in these three screenshots:

[jimshowalter]

Was able to work around this by creating a repo that only has the bouncy-castle dependencies, shadowing and publishing that, and use it in the original repo.

[jimshowalter]

Forgot to mention that minimize() has no effect.

[ejjcase]

I believe this occurs when java-gradle-plugin is applied.

The manual recommends adding dependencies to the shadow configuration:

shadow(gradleApi())
shadow(localGroovy())

but this had no effect in my project. It used to work with Shadow 6.1.0 and Gradle 6.8.3 (I found I didn't even need the localGroovy() dependency, since my plugin is written in Kotlin).

I worked around it by setting an include on the shadowJar task. This needed to cover the Gradle plugin metadata and Kotlin module metadata too:

tasks.shadowJar {
  include(
    "com/mycompany/**/*",
    "my-dependency-*",
    "META-INF/gradle-plugins/com.mycompany.*",
    "META-INF/my-project.kotlin_module",
    "META-INF/my-dependency.kotlin_module"
  )
}

to only include the project's own classes and the one dependency I wanted to bundle.

[tsjensen]

Same problem here, using Gradle 8.9 and the java-gradle-plugin, with shadow 8.3.0.

@jimshowalter's workaround would work, but it's painful of course.

I noticed that the Problem does not appear on Gradle < 8, and also the old jonrengelman plugin works fine even with Gradle 8 (but with it's own limitations, of course).

@Goooler Can you have a look at this one of these days? That would be fantastic, as it's currently blocking us from using the new plugin. Thanks!!

[Goooler]

Fixed in #948.

[tsjensen]

Wow, that's great! Thank you!

[Goooler]

https://github.com/GradleUp/shadow/releases/tag/8.3.1