[MDEP-753] Non-test dependency reported as Non-test scoped test only dependency

[jira-importer]

Elliotte Rusty Harold opened MDEP-753 and commented

Saw this when updating the google-http-java-client from 3.1.2 to 3.2.0 of the plugin. I'm not immediately sure whether this is a regression:

[INFO] --- maven-dependency-plugin:3.2.0:analyze (default-cli) @ google-http-client ---
Warning: Non-test scoped test only dependencies found:
Warning: com.google.guava:guava:jar:30.1.1-android:compile
Warning: io.opencensus:opencensus-api:jar:0.28.0:compile

Changing Guava to scope test breaks the build, which is expected based on the code. The warning seems incorrect.

googleapis/google-http-java-client#1396
https://github.com/googleapis/google-http-java-client/pull/1396/checks?check_run_id=2809438131

---

Affects: 3.2.0

Attachments:

• chas.zip (1.69 kB)
• tj.zip (1.40 kB)

Issue Links:

• MSHARED-1021 Handle different classes from same artifact used by model and test code
  ("is blocked by")

• MDEP-784 Upgrade maven-dependency-analyzer to 1.12.0
  ("is blocked by")

• MDEP-791 Non-test scoped and transitive dependencies in compile scope
  ("split to")

• MDEP-790 New Goals: analyze-main and analyze-test
  ("split to")

• MDEP-757 bug with "non-test scoped test only dependencies found"

• MDEP-759 'Dependency not found' with 3.2.0 and Java-17 while analyzing

• MDEP-780 “Non-test scoped test only dependencies found” false-positive in 3.2.0/1.11.2

• MDEP-754 Analyze incorrectly warns that junit is test only when it is a compile time dependency

• MDEP-787 Allow ignoring non-test-scoped dependencies

Remote Links:

• GitHub Pull Request #34
  
• GitHub Pull Request #35
  
• GitHub Pull Request #37
  

40 votes, 45 watchers

[jira-importer]

Joe Barnett commented

We're seeing this as well with code that is definitely using the listed dependencies in non-test code, would say this is both a bug and a regression. 

https://app.circleci.com/pipelines/github/trib3/leakycauldron/2100/workflows/58469b8d-0464-4733-b980-fea291e8076f/jobs/2190

[jira-importer]

Steven Schlansker commented

I attempted to upgrade our project to use Java 16 source level and when we upgraded the dependency-analyzer, we also ran into this problem. For us it is commons-lang3 which is obviously used in main sources, but triggers a failure due to this check.

I would recommend raising the priority of this bug, potentially even to Blocker, this is a serious regression.

[jira-importer]

Elliotte Rusty Harold commented

It seems likely the bug lies in org.apache.maven.shared.dependency.analyzer.DefaultProjectDependencyAnalyzer

[jira-importer]

Elliotte Rusty Harold commented

maven-dependency-plugin 3.1.2 used maven-dependency-analyzer 1.11.1
maven-dependency-plugin 3.2.0 uses maven-dependency-analyzer 1.11.3

[jira-importer]

Elliotte Rusty Harold commented

What we really need to fix this is a super simple project that exhibits the problem suitable for use in an integration test.

[jira-importer]

Elliotte Rusty Harold commented

Just an educated guess but this might be the problem:

{{ private Set<Artifact> getTestArtifactsWithNonTestScope( Set<Artifact> testOnlyArtifacts )
{
Set<Artifact> nonTestScopeArtifacts = new LinkedHashSet<>();

    for ( Artifact artifact : testOnlyArtifacts )
    {
        if ( artifact.getScope().equals( "compile" ) )
        {
            nonTestScopeArtifacts.add( artifact );
        }
    }

    return nonTestScopeArtifacts;
}}}

In particular is it possible that artifact.getScope().equals( "compile" ) misses cases where the scope is null or empty?

[jira-importer]

Sergey Vyacheslavovich Brunov commented

Dear Steven Schlansker and Elliotte Rusty Harold,

Recently, I faced the same problem with the upgraded plugin (3.2.0) and was going to report it.

What we really need to fix this is a super simple project that exhibits the problem suitable for use in an integration test.

Elliotte Rusty Harold, I have created such a MVCE (Minimal, Complete, and Verifiable example) project:

• Branch-based link: maven-dependency-plugin-examples/issues.apache.org/MDEP-753 at master · svbrunov/maven-dependency-plugin-examples.
• Commit-based link: maven-dependency-plugin-examples/issues.apache.org/MDEP-753 at master · svbrunov/maven-dependency-plugin-examples.

Please, consider using this project as a starting point.

The build fails with the error message:

[INFO] --- maven-dependency-plugin:3.2.0:analyze-only (analyze) @ maven-example ---
[INFO] Used declared dependencies found:
[INFO]    com.google.guava:guava:jar:30.1.1-jre:compile
[INFO]    org.junit.jupiter:junit-jupiter-api:jar:5.8.0-M1:test
[INFO]    org.junit.jupiter:junit-jupiter-params:jar:5.8.0-M1:test
[WARNING] Non-test scoped test only dependencies found:
[WARNING]    com.google.guava:guava:jar:30.1.1-jre:compile
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------

<…>
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:3.2.0:analyze-only (analyze) on project maven-example: Dependency problems found -> [Help 1]
[ERROR] 

Please, notice that it seems the problem occurs because the production code uses one class from Guava, while the test code uses another (different) class from Guava.

I would recommend raising the priority of this bug, potentially even to Blocker, this is a serious regression.

I agree with Steven Schlansker because it is a serious regression defect.
I had to downgrade the plugin to the 3.1.2 version.

Best regards,
Sergey Vyacheslavovich Brunov.

[jira-importer]

Elliotte Rusty Harold commented

I think you nailed it. The problem is "the production code uses one class from Guava, while the test code uses another (different) class from Guava." I see where the mistake in the code is, and that explains why some projects encounter this and one don't.

Testing this is going to be tricky.

[jira-importer]

Elliotte Rusty Harold commented

Thanks. That does illustrate the problem. It's not minimal enough to run inside the dependency analyzer tests though. I haven't been able to reproduce it in that context yet.

[jira-importer]

Elliotte Rusty Harold commented

I've committed a fix for the bug. It now needs a release of the maven-dependency-analyzer and an upgrade of the maven-dependency-plugin to the fixed version of the analyzer.

[jira-importer]

Henning Schmiedehausen commented

I opened MDEP-757 which is a potential duplicate (except that my issue is specifically with annotations). I linked a minimal project to reproduce the problem here: https://github.com/hgschmie/mdep757

[jira-importer]

Sergey Vyacheslavovich Brunov commented

Dear Elliotte Rusty Harold,

I've committed a fix for the bug. It now needs a release of the maven-dependency-analyzer and an upgrade of the maven-dependency-plugin to the fixed version of the analyzer.

This is great.

Could you please provide us with the estimated release date of maven-dependency-plugin?

Best regards,
Sergey Vyacheslavovich Brunov.

[jira-importer]

Henning Schmiedehausen commented

Elliotte Rusty Harold - I pulled down the trunk of the maven-dependency-analyzer from git@github.com:apache/maven-dependency-analyzer.git, built and installed it (the tests don't pass for me BTW).

I then ran this as follows:

<plugin>
  <artifactId>maven-dependency-plugin</artifactId>
  <version>3.2.0</version>
  <dependencies>
    <dependency>
      <groupId>org.apache.maven.shared</groupId>
      <artifactId>maven-dependency-analyzer</artifactId>
      <version>1.11.4-SNAPSHOT</version>
    </dependency>
  </dependencies>
  <configuration>
    <failOnWarning>true</failOnWarning>
    <ignoreNonCompile>true</ignoreNonCompile>
  </configuration>
</plugin>

and was greeted with this:

 

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:3.2.0:analyze-only (basepom.default) on project testcase: Execution basepom.default of goal org.apache.maven.plugins:maven-dependency-plugin:3.2.0:analyze-only failed: An API incompatibility was encountered while executing org.apache.maven.plugins:maven-dependency-plugin:3.2.0:analyze-only: java.lang.NoSuchFieldError: ROLE
[ERROR] -----------------------------------------------------
[ERROR] realm = plugin>org.apache.maven.plugins:maven-dependency-plugin:3.2.0
[ERROR] strategy = org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy
[ERROR] urls[0] = file:/Users/henning/.m2/repository/org/apache/maven/plugins/maven-dependency-plugin/3.2.0/maven-dependency-plugin-3.2.0.jar
[ERROR] urls[1] = file:/Users/henning/.m2/repository/org/apache/maven/shared/maven-dependency-analyzer/1.11.4-SNAPSHOT/maven-dependency-analyzer-1.11.4-SNAPSHOT.jar
[ERROR] urls[2] = file:/Users/henning/.m2/repository/org/ow2/asm/asm/9.1/asm-9.1.jar
[ERROR] urls[3] = file:/Users/henning/.m2/repository/org/codehaus/plexus/plexus-component-annotations/2.0.0/plexus-component-annotations-2.0.0.jar
[ERROR] urls[4] = file:/Users/henning/.m2/repository/javax/enterprise/cdi-api/1.0/cdi-api-1.0.jar
[ERROR] urls[5] = file:/Users/henning/.m2/repository/com/google/guava/guava/10.0.1/guava-10.0.1.jar
[ERROR] urls[6] = file:/Users/henning/.m2/repository/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar
[ERROR] urls[7] = file:/Users/henning/.m2/repository/org/sonatype/sisu/sisu-guice/3.1.0/sisu-guice-3.1.0-no_aop.jar
[ERROR] urls[8] = file:/Users/henning/.m2/repository/aopalliance/aopalliance/1.0/aopalliance-1.0.jar
[ERROR] urls[9] = file:/Users/henning/.m2/repository/org/eclipse/sisu/org.eclipse.sisu.inject/0.0.0.M5/org.eclipse.sisu.inject-0.0.0.M5.jar
[ERROR] urls[10] = file:/Users/henning/.m2/repository/org/codehaus/plexus/plexus-interpolation/1.19/plexus-interpolation-1.19.jar
[ERROR] urls[11] = file:/Users/henning/.m2/repository/org/sonatype/plexus/plexus-sec-dispatcher/1.3/plexus-sec-dispatcher-1.3.jar
[ERROR] urls[12] = file:/Users/henning/.m2/repository/org/sonatype/plexus/plexus-cipher/1.4/plexus-cipher-1.4.jar
[ERROR] urls[13] = file:/Users/henning/.m2/repository/org/apache/maven/reporting/maven-reporting-api/3.0/maven-reporting-api-3.0.jar
[ERROR] urls[14] = file:/Users/henning/.m2/repository/org/apache/maven/reporting/maven-reporting-impl/3.0.0/maven-reporting-impl-3.0.0.jar
[ERROR] urls[15] = file:/Users/henning/.m2/repository/org/apache/maven/doxia/doxia-decoration-model/1.7.4/doxia-decoration-model-1.7.4.jar
[ERROR] urls[16] = file:/Users/henning/.m2/repository/commons-io/commons-io/2.6/commons-io-2.6.jar
[ERROR] urls[17] = file:/Users/henning/.m2/repository/org/apache/maven/doxia/doxia-sink-api/1.9.1/doxia-sink-api-1.9.1.jar
[ERROR] urls[18] = file:/Users/henning/.m2/repository/org/apache/maven/doxia/doxia-logging-api/1.9.1/doxia-logging-api-1.9.1.jar
[ERROR] urls[19] = file:/Users/henning/.m2/repository/org/apache/maven/doxia/doxia-core/1.9.1/doxia-core-1.9.1.jar
[ERROR] urls[20] = file:/Users/henning/.m2/repository/org/apache/xbean/xbean-reflect/3.7/xbean-reflect-3.7.jar
[ERROR] urls[21] = file:/Users/henning/.m2/repository/com/google/collections/google-collections/1.0/google-collections-1.0.jar
[ERROR] urls[22] = file:/Users/henning/.m2/repository/org/apache/httpcomponents/httpclient/4.5.8/httpclient-4.5.8.jar
[ERROR] urls[23] = file:/Users/henning/.m2/repository/commons-logging/commons-logging/1.2/commons-logging-1.2.jar
[ERROR] urls[24] = file:/Users/henning/.m2/repository/commons-codec/commons-codec/1.11/commons-codec-1.11.jar
[ERROR] urls[25] = file:/Users/henning/.m2/repository/org/apache/httpcomponents/httpcore/4.4.11/httpcore-4.4.11.jar
[ERROR] urls[26] = file:/Users/henning/.m2/repository/org/apache/maven/doxia/doxia-site-renderer/1.9.2/doxia-site-renderer-1.9.2.jar
[ERROR] urls[27] = file:/Users/henning/.m2/repository/org/apache/maven/doxia/doxia-skin-model/1.9.2/doxia-skin-model-1.9.2.jar
[ERROR] urls[28] = file:/Users/henning/.m2/repository/org/apache/maven/doxia/doxia-module-xhtml/1.9.1/doxia-module-xhtml-1.9.1.jar
[ERROR] urls[29] = file:/Users/henning/.m2/repository/org/apache/maven/doxia/doxia-module-xhtml5/1.9.1/doxia-module-xhtml5-1.9.1.jar
[ERROR] urls[30] = file:/Users/henning/.m2/repository/org/codehaus/plexus/plexus-i18n/1.0-beta-10/plexus-i18n-1.0-beta-10.jar
[ERROR] urls[31] = file:/Users/henning/.m2/repository/org/codehaus/plexus/plexus-velocity/1.2/plexus-velocity-1.2.jar
[ERROR] urls[32] = file:/Users/henning/.m2/repository/org/apache/velocity/velocity/1.7/velocity-1.7.jar
[ERROR] urls[33] = file:/Users/henning/.m2/repository/commons-lang/commons-lang/2.4/commons-lang-2.4.jar
[ERROR] urls[34] = file:/Users/henning/.m2/repository/org/apache/velocity/velocity-tools/2.0/velocity-tools-2.0.jar
[ERROR] urls[35] = file:/Users/henning/.m2/repository/commons-beanutils/commons-beanutils/1.7.0/commons-beanutils-1.7.0.jar
[ERROR] urls[36] = file:/Users/henning/.m2/repository/commons-digester/commons-digester/1.8/commons-digester-1.8.jar
[ERROR] urls[37] = file:/Users/henning/.m2/repository/commons-chain/commons-chain/1.1/commons-chain-1.1.jar
[ERROR] urls[38] = file:/Users/henning/.m2/repository/dom4j/dom4j/1.1/dom4j-1.1.jar
[ERROR] urls[39] = file:/Users/henning/.m2/repository/oro/oro/2.0.8/oro-2.0.8.jar
[ERROR] urls[40] = file:/Users/henning/.m2/repository/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar
[ERROR] urls[41] = file:/Users/henning/.m2/repository/org/codehaus/plexus/plexus-archiver/4.2.2/plexus-archiver-4.2.2.jar
[ERROR] urls[42] = file:/Users/henning/.m2/repository/org/apache/commons/commons-compress/1.20/commons-compress-1.20.jar
[ERROR] urls[43] = file:/Users/henning/.m2/repository/org/iq80/snappy/snappy/0.4/snappy-0.4.jar
[ERROR] urls[44] = file:/Users/henning/.m2/repository/org/tukaani/xz/1.8/xz-1.8.jar
[ERROR] urls[45] = file:/Users/henning/.m2/repository/org/codehaus/plexus/plexus-utils/3.3.0/plexus-utils-3.3.0.jar
[ERROR] urls[46] = file:/Users/henning/.m2/repository/org/codehaus/plexus/plexus-io/3.2.0/plexus-io-3.2.0.jar
[ERROR] urls[47] = file:/Users/henning/.m2/repository/org/apache/maven/shared/maven-dependency-tree/3.0.1/maven-dependency-tree-3.0.1.jar
[ERROR] urls[48] = file:/Users/henning/.m2/repository/org/apache/maven/shared/maven-common-artifact-filters/3.2.0/maven-common-artifact-filters-3.2.0.jar
[ERROR] urls[49] = file:/Users/henning/.m2/repository/org/apache/maven/shared/maven-artifact-transfer/0.13.1/maven-artifact-transfer-0.13.1.jar
[ERROR] urls[50] = file:/Users/henning/.m2/repository/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.jar
[ERROR] urls[51] = file:/Users/henning/.m2/repository/org/apache/commons/commons-lang3/3.8.1/commons-lang3-3.8.1.jar
[ERROR] urls[52] = file:/Users/henning/.m2/repository/org/apache/commons/commons-collections4/4.2/commons-collections4-4.2.jar
[ERROR] urls[53] = file:/Users/henning/.m2/repository/org/eclipse/aether/aether-util/0.9.0.M2/aether-util-0.9.0.M2.jar
[ERROR] Number of foreign imports: 1
[ERROR] import: Entry[import from realm ClassRealm[maven.api, parent: null]]
[ERROR]
[ERROR] -----------------------------------------------------

 

So it seems to me that there also needs to be a new release of the plugin itself as the analyzer API is different. (To paraphrase you from MSHARED-957: "maven-dependency-plugin 3.2.0 declares a dependency on analyzer 1.11.3 and it works with that version. There is no expectation that 3.2.0 works with any other version of the dependency analyzer and failure to do so is not a bug.")

I tried to build the current master of the maven-dependency-plugin as well with the 1.11.4-SNAPSHOT of the maven-dependency-analyzer and I received 

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 16.725 s
[INFO] Finished at: 2021-06-22T17:51:16-07:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) on project maven-dependency-plugin: Compilation failure
[ERROR] /Users/henning/scratch/maven-dependency-plugin/src/main/java/org/apache/maven/plugins/dependency/analyze/AbstractAnalyzeMojo.java:[268,54] cannot find symbol
[ERROR] symbol: variable ROLE
[ERROR] location: interface org.apache.maven.shared.dependency.analyzer.ProjectDependencyAnalyzer

 

So it seems that these are really not just drop-in compatible.

 

 

[jira-importer]

Henning Schmiedehausen commented

Is there any progress on this or something that we can test / try out?

[jira-importer]

Sergey Vyacheslavovich Brunov commented

Dear Elliotte Rusty Harold and Robert Scholte,

As it was highlighted earlier, this defect is critical.
Therefore, it would be great to have it fixed and to release the fixed version.

Could you please provide us with the estimated release date of maven-dependency-plugin?

Thank you in advance!

Best regards,
Sergey Vyacheslavovich Brunov.