Issue resolving merged jars with test suites

[crschnick]

I have configured the following test suite:

testing {
    suites {
        localTest(JvmTestSuite) {
            useJUnitJupiter()

            dependencies {
                implementation project(':app')
                def imp = project(':app').getConfigurations().getAsMap().get("implementation")
                if (imp) {
                    imp.dependencies.forEach {
                        implementation it
                    }
                }
            }
        }
    }
}

that I use in several gradle modules to have the app project and its dependencies available when running the test. This works fine for regular dependencies.

However, I also have this extra module info defined:

extraJavaModuleInfo {
    module("com.vladsch.flexmark:flexmark", "com.vladsch.flexmark") {
        mergeJar('com.vladsch.flexmark:flexmark-util-data')
        mergeJar('com.vladsch.flexmark:flexmark-util-format')
        mergeJar('com.vladsch.flexmark:flexmark-util-ast')
        mergeJar('com.vladsch.flexmark:flexmark-util-sequence')
        mergeJar('com.vladsch.flexmark:flexmark-util-builder')
        mergeJar('com.vladsch.flexmark:flexmark-util-html')
        mergeJar('com.vladsch.flexmark:flexmark-util-dependency')
        mergeJar('com.vladsch.flexmark:flexmark-util-collection')
        mergeJar('com.vladsch.flexmark:flexmark-util-misc')
        mergeJar('com.vladsch.flexmark:flexmark-util-visitor')
        exportAllPackages()
    }
}

In the app project, I have all dependencies added:

dependencies {
    implementation 'com.vladsch.flexmark:flexmark:0.64.0'
    implementation 'com.vladsch.flexmark:flexmark-util-data:0.64.0'
    implementation 'com.vladsch.flexmark:flexmark-util-ast:0.64.0'
    implementation 'com.vladsch.flexmark:flexmark-util-builder:0.64.0'
    implementation 'com.vladsch.flexmark:flexmark-util-sequence:0.64.0'
    implementation 'com.vladsch.flexmark:flexmark-util-misc:0.64.0'
    implementation 'com.vladsch.flexmark:flexmark-util-dependency:0.64.0'
    implementation 'com.vladsch.flexmark:flexmark-util-collection:0.64.0'
    implementation 'com.vladsch.flexmark:flexmark-util-format:0.64.0'
    implementation 'com.vladsch.flexmark:flexmark-util-html:0.64.0'
    implementation 'com.vladsch.flexmark:flexmark-util-visitor:0.64.0'
}

When I try to run the localTest suite in another module, lets call it the ext gradle module, which uses the app module for tests in its test suite, I get the following error:

Execution failed for task ':ext:compileLocalTestJava'.
> Could not resolve all files for configuration ':ext:localTestCompileClasspath'.
   > Failed to transform flexmark-0.64.0.jar (com.vladsch.flexmark:flexmark:0.64.0) to match attributes {artifactType=jar, javaModule=true, org.gradle.category=library, org.gradle.libraryelements=jar, org.gradle.status=release, org.gradle.usage=java-api}.
      > Execution failed for ExtraJavaModuleInfoTransform: C:\Users\Christopher Schnick\.gradle\caches\modules-2\files-2.1\com.vladsch.flexmark\flexmark\0.64.0\bb5fcdf1335a35c4c0285fee2683a32e6a70cd59\flexmark-0.64.0.jar.
         > Jar not found: com.vladsch.flexmark:flexmark-util-data

However, if I add only one dependency to the ext gradle module, it fixes the problem:

dependencies {
    implementation 'com.vladsch.flexmark:flexmark-util-data:0.64.0'
}

This behavior is a little bit unexpected. I either expected it to require me to define all dependencies again or work without having to do that. I'm not sure whether that is a bug though.

[jjohannes]

This looks to me like a general Gradle configuration timing problem.

In your setup, you directly access the configuration time state of another (sub)project. This is generally discouraged and future Gradle versions might even forbid it.

def imp = project(':app').getConfigurations().getAsMap().get("implementation")
if (imp) {
  imp.dependencies.forEach {
    implementation it
  }
}

When you access the "implementation" Configuration of the ":app" project, it might not yet be filled with dependencies if that part of the ":app" project is configured after the ":ext" project.

I know too little about your project, to give you the exact advice for how to do this better. Maybe the dependencies in ":app" can be defined in "api" scope? Then they would be automatically visible at compile time by this line alone:
implementation project(':app')

[crschnick]

Thanks for the explanation, I changed the :app dependencies to the following:

dependencies {
    api 'com.vladsch.flexmark:flexmark:0.64.0'
    api 'com.vladsch.flexmark:flexmark-util-data:0.64.0'
    api 'com.vladsch.flexmark:flexmark-util-ast:0.64.0'
    api 'com.vladsch.flexmark:flexmark-util-builder:0.64.0'
    api 'com.vladsch.flexmark:flexmark-util-sequence:0.64.0'
    api 'com.vladsch.flexmark:flexmark-util-misc:0.64.0'
    api 'com.vladsch.flexmark:flexmark-util-dependency:0.64.0'
    api 'com.vladsch.flexmark:flexmark-util-collection:0.64.0'
    api 'com.vladsch.flexmark:flexmark-util-format:0.64.0'
    api 'com.vladsch.flexmark:flexmark-util-html:0.64.0'
    api 'com.vladsch.flexmark:flexmark-util-visitor:0.64.0'
}

and removed all instances of the configuration access. To make this case simpler, I also removed the test suite for reproducing this issue.

The only thing I now have in the :ext project is the following:

dependencies {
    compileOnly project(':app')
}

and I still get the following error:

Execution failed for task ':ext:compileJava'.
> Could not resolve all files for configuration ':ext:compileClasspath'.
   > Failed to transform flexmark-0.64.0.jar (com.vladsch.flexmark:flexmark:0.64.0) to match attributes {artifactType=jar, javaModule=true, org.gradle.category=library, org.gradle.libraryelements=jar, org.gradle.status=release, org.gradle.usage=java-api}.
      > Execution failed for ExtraJavaModuleInfoTransform: C:\Users\Christopher Schnick\.gradle\caches\modules-2\files-2.1\com.vladsch.flexmark\flexmark\0.64.0\bb5fcdf1335a35c4c0285fee2683a32e6a70cd59\flexmark-0.64.0.jar.
         > Jar not found: com.vladsch.flexmark:flexmark-util-data

If I change the ext project to

dependencies {
    implementation project(':app')
}

it works however.

[jjohannes]

Ah yes. This a specific tidbit with the "jar merge" functionality. To perform the merge in isolation, there is a specific dependency scope – configurations.javaModulesMergeJars – to which the plugin adds dependencies that should be merged. However, it does not have any versions for these dependencies.

The plugin does:

configurations.javaModulesMergeJars.get().shouldResolveConsistentlyWith(configurations.runtimeClasspath)

Which means, it takes versions from the runtime classpath – which works in most cases. But if you only use compileOnly, the libraries are not (all) on the runtime classpath. Maybe this can be improved, but I am afraid changing something in the implementation will break other cases. It is complex, as the Jars you merge are not necessary alway all on your classpath if you would use them independently.

I can check if I can add some more information to the docs and/or improve the error message.

What you can do, if you want to keep the compileOnly:

configurations.javaModulesMergeJars.shouldResolveConsistentlyWith(configurations.compileClasspath)

Or see set the javaModulesMergeJars scope gets the version information from somewhere else. E.g. by defining constraints or a dependency to a platform if that is something you do already as central version management in your build.

dependency.constraints {
  javaModulesMergeJars(...)
}

[crschnick]

Thanks for the help, it works now as expected!

So yeah I think adding a little bit more detail to the error messages or documentation would be helpful in this case also for other people as I'm probably not the only one who uses compileOnly for some merged module dependencies.

[jjohannes]

Follow up: #107