grpc-auth's unit test fails with google-auth-library 0.25.X

[suztomo]

Background: I'm surveying library compatibility across Google Java libraries.

I haven't figured out whether this is a bug in grpc-auth or google-auth-library.

What version of gRPC-Java are you using?

1.36.1

What is your environment?

Mac

What did you expect to see?

All tests pass

What did you see instead?

./gradlew :grpc-auth:test fails:

suztomo-macbookpro44% ./gradlew :grpc-auth:test
*** Skipping the build of codegen and compilation of proto files because skipCodegen=true
  * Skipping the build of Android projects because skipAndroid=true

> Task :grpc-auth:test

io.grpc.auth.GoogleAuthLibraryCallCredentialsTest > jwtAccessCredentialsInRequestMetadata FAILED
    org.junit.ComparisonFailure: expected:<https://example.com[:123/a.service]> but was:<https://example.com[/]>
        at org.junit.Assert.assertEquals(Assert.java:115)
        at org.junit.Assert.assertEquals(Assert.java:144)
        at io.grpc.auth.GoogleAuthLibraryCallCredentialsTest.jwtAccessCredentialsInRequestMetadata(GoogleAuthLibraryCallCredentialsTest.java:397)

16 tests completed, 1 failed

> Task :grpc-auth:test FAILED

Steps to reproduce the bug

Upgrade the version of googleauthVersion.

suztomo-macbookpro44% git diff                 
diff --git a/build.gradle b/build.gradle
index 9baf8e029..81fbc4d86 100644
--- a/build.gradle
+++ b/build.gradle
@@ -56,7 +56,7 @@ subprojects {
 
         nettyVersion = '4.1.52.Final'
         guavaVersion = '30.0-android'
-        googleauthVersion = '0.22.2'
+        googleauthVersion = '0.25.2'
         protobufVersion = '3.12.0'
         protocVersion = protobufVersion
         opencensusVersion = '0.28.0'

The variable is used in

            google_auth_credentials: "com.google.auth:google-auth-library-credentials:${googleauthVersion}",
            google_auth_oauth2_http: "com.google.auth:google-auth-library-oauth2-http:${googleauthVersion}",

Since when it doesn't work?

google-auth-library version	:grpc-auth:test success/failure
0.25.2	fail
0.25.1	fail
0.25.0	fail
0.24.1	success
0.25.0	success

It seems the problem started after google-auth-library 0.25.0.

[suztomo]

As per googleapis/google-auth-library-java#620, it's expected change.

IntelliJ complains that the project also gets dependency conflicts;

suztomo-macbookpro44% ./gradlew     :grpc-all:dependencyInsight --configuration runtimeClasspath --dependency org.apache.httpcomponents:httpcore                                                                                 
*** Skipping the build of codegen and compilation of proto files because skipCodegen=true
  * Skipping the build of Android projects because skipAndroid=true

> Task :grpc-all:dependencyInsight
Dependency resolution failed because of conflict(s) on the following module(s):
   - org.apache.httpcomponents:httpcore between versions 4.4.14 and 4.4.13

org.apache.httpcomponents:httpcore:4.4.14
   variant "runtime" [
      org.gradle.status              = release (not requested)
      org.gradle.usage               = java-runtime
      org.gradle.libraryelements     = jar
      org.gradle.category            = library

      Requested attributes not found in the selected variant:
         org.gradle.dependency.bundling = external
         org.gradle.jvm.version         = 7
   ]
   Selection reasons:
      - By conflict resolution : between versions 4.4.14 and 4.4.13

org.apache.httpcomponents:httpcore:4.4.14
\--- com.google.http-client:google-http-client:1.39.1
     +--- com.google.auth:google-auth-library-oauth2-http:0.25.2
     |    \--- project :grpc-alts
     |         \--- project :grpc-xds
     |              \--- runtimeClasspath
     \--- com.google.http-client:google-http-client-gson:1.39.1
          \--- com.google.auth:google-auth-library-oauth2-http:0.25.2 (*)

org.apache.httpcomponents:httpcore:4.4.13 -> 4.4.14
\--- org.apache.httpcomponents:httpclient:4.5.13
     \--- com.google.http-client:google-http-client:1.39.1
          +--- com.google.auth:google-auth-library-oauth2-http:0.25.2
          |    \--- project :grpc-alts
          |         \--- project :grpc-xds
          |              \--- runtimeClasspath
          \--- com.google.http-client:google-http-client-gson:1.39.1
               \--- com.google.auth:google-auth-library-oauth2-http:0.25.2 (*)

com.google.http-client:google-http-client:1.39.1 is the latest as of now. httpclient:4.5.13 is also latest but it has httpcore:4.4.13.

Problem is that com.google.http-client:google-http-client:1.39.1's dependency does not have dependency convergence.

• com.google.http-client:google-http-client:1.39.1 -> org.apache.httpcomponents:httpclient:4.5.13 -> org.apache.httpcomponents:httpcore:4.4.13
• com.google.http-client:google-http-client:1.39.1 -> org.apache.httpcomponents:httpcore:4.4.14

It's not gRPC's dependency or google-auth-library-oauth2-http. Any gradle project with failOnVersionConflict cannot declare dependency with com.google.http-client:google-http-client:1.39.1

Apache HTTP Client team does not have upcoming release for 4.X release (I asked mailing list)

[dapengzhang0]

The test need be updated once google-auth dependency is updated. Thanks @suztomo

[suztomo]

Another problem

> Task :grpc-xds:compileJava FAILED
/Users/suztomo/grpc-java/xds/src/main/java/io/grpc/xds/internal/sts/StsCredentials.java:29: error: package com.google.api.client.json.jackson2 does not exist
import com.google.api.client.json.jackson2.JacksonFactory;
                                          ^
1 error

FAILURE: Build failed with an exception.

com.google.api.client.json.jackson2.JacksonFactory is in google-http-client-jackson2:1.38.0.

[suztomo]

When I add google-http-client-jackson2 dependency (f282366), it still failed over 3 dependency conflicts.

suztomo-macbookpro44%  ./gradlew assemble                                      
*** Skipping the build of codegen and compilation of proto files because skipCodegen=true
  * Skipping the build of Android projects because skipAndroid=true

FAILURE: Build failed with an exception.

* What went wrong:
Could not determine the dependencies of task ':grpc-xds:shadowJar'.
> Could not resolve all dependencies for configuration ':grpc-xds:runtimeClasspath'.
   > Conflict(s) found for the following module(s):
       - com.google.errorprone:error_prone_annotations between versions 2.4.0 and 2.3.4
       - io.grpc:grpc-context between versions 1.38.0-SNAPSHOT and 1.27.2
       - com.google.guava:guava between versions 30.1-android, 30.0-android and 29.0-android
     Run with:
         --scan or
         :grpc-xds:dependencyInsight --configuration runtimeClasspath --dependency com.google.errorprone:error_prone_annotations
     to get more insight on how to solve the conflict.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 1s

com.google.errorprone:error_prone_annotations:

suztomo-macbookpro44% ./gradlew :grpc-xds:dependencyInsight --configuration runtimeClasspath --dependency com.google.errorprone:error_prone_annotations
*** Skipping the build of codegen and compilation of proto files because skipCodegen=true
  * Skipping the build of Android projects because skipAndroid=true

> Task :grpc-xds:dependencyInsight
Dependency resolution failed because of conflict(s) on the following module(s):
   - com.google.errorprone:error_prone_annotations between versions 2.4.0 and 2.3.4

com.google.errorprone:error_prone_annotations:2.4.0
   variant "runtime" [
      org.gradle.status              = release (not requested)
      org.gradle.usage               = java-runtime
      org.gradle.libraryelements     = jar
      org.gradle.category            = library

      Requested attributes not found in the selected variant:
         org.gradle.dependency.bundling = external
         org.gradle.jvm.version         = 7
   ]
   Selection reasons:
      - By conflict resolution : between versions 2.4.0 and 2.3.4

com.google.errorprone:error_prone_annotations:2.4.0
+--- project :grpc-alts
|    \--- runtimeClasspath
+--- project :grpc-api
|    +--- project :grpc-protobuf
|    |    +--- runtimeClasspath
|    |    +--- project :grpc-services
|    |    |    \--- runtimeClasspath
|    |    +--- project :grpc-alts (*)
|    |    \--- project :grpc-grpclb
|    |         \--- project :grpc-alts (*)
|    +--- project :grpc-stub
|    |    +--- runtimeClasspath
|    |    +--- project :grpc-services (*)
|    |    +--- project :grpc-alts (*)
|    |    \--- project :grpc-grpclb (*)
|    +--- project :grpc-core
|    |    +--- runtimeClasspath
|    |    +--- project :grpc-services (*)
|    |    +--- project :grpc-grpclb (*)
|    |    +--- project :grpc-netty
|    |    |    \--- runtimeClasspath
|    |    \--- project :grpc-netty-shaded
|    |         \--- project :grpc-alts (*)
|    +--- project :grpc-auth
|    |    +--- runtimeClasspath
|    |    \--- project :grpc-alts (*)
|    \--- project :grpc-protobuf-lite
|         \--- project :grpc-protobuf (*)
+--- project :grpc-auth (*)
+--- project :grpc-core (*)
+--- project :grpc-grpclb (*)
+--- project :grpc-netty (*)
+--- project :grpc-protobuf (*)
+--- project :grpc-protobuf-lite (*)
+--- project :grpc-services (*)
\--- project :grpc-stub (*)

com.google.errorprone:error_prone_annotations:2.3.4 -> 2.4.0
\--- com.google.guava:guava:30.1-android
     +--- project :grpc-services (requested com.google.guava:guava:30.0-android)
     |    \--- runtimeClasspath
     +--- project :grpc-alts (requested com.google.guava:guava:30.0-android)
     |    \--- runtimeClasspath
     +--- project :grpc-grpclb (requested com.google.guava:guava:30.0-android)
     |    \--- project :grpc-alts (*)
     +--- project :grpc-protobuf (requested com.google.guava:guava:30.0-android)
     |    +--- runtimeClasspath
     |    +--- project :grpc-services (*)
     |    +--- project :grpc-alts (*)
     |    \--- project :grpc-grpclb (*)
     +--- project :grpc-stub (requested com.google.guava:guava:30.0-android)
     |    +--- runtimeClasspath
     |    +--- project :grpc-services (*)
     |    +--- project :grpc-alts (*)
     |    \--- project :grpc-grpclb (*)
     +--- project :grpc-netty (requested com.google.guava:guava:30.0-android)
     |    \--- runtimeClasspath
     +--- project :grpc-core (requested com.google.guava:guava:30.0-android)
     |    +--- runtimeClasspath
     |    +--- project :grpc-services (*)
     |    +--- project :grpc-grpclb (*)
     |    +--- project :grpc-netty (*)
     |    \--- project :grpc-netty-shaded
     |         \--- project :grpc-alts (*)
     +--- project :grpc-auth (requested com.google.guava:guava:30.0-android)
     |    +--- runtimeClasspath
     |    \--- project :grpc-alts (*)
     +--- project :grpc-protobuf-lite (requested com.google.guava:guava:30.0-android)
     |    \--- project :grpc-protobuf (*)
     +--- project :grpc-api (requested com.google.guava:guava:30.0-android)
     |    +--- project :grpc-protobuf (*)
     |    +--- project :grpc-stub (*)
     |    +--- project :grpc-core (*)
     |    +--- project :grpc-auth (*)
     |    \--- project :grpc-protobuf-lite (*)
     +--- com.google.http-client:google-http-client:1.39.1
     |    +--- com.google.auth:google-auth-library-oauth2-http:0.25.2
     |    |    \--- project :grpc-alts (*)
     |    +--- com.google.http-client:google-http-client-gson:1.39.1
     |    |    \--- com.google.auth:google-auth-library-oauth2-http:0.25.2 (*)
     |    \--- com.google.http-client:google-http-client-jackson2:1.39.1
     |         \--- runtimeClasspath
     \--- io.opencensus:opencensus-contrib-http-util:0.28.0 (requested com.google.guava:guava:29.0-android)
          \--- com.google.http-client:google-http-client:1.39.1 (*)

(*) - dependencies omitted (listed previously)

A web-based, searchable dependency report is available by adding the --scan option.

BUILD SUCCESSFUL in 1s
1 actionable task: 1 executed

grpc-context:

suztomo-macbookpro44% ./gradlew :grpc-xds:dependencyInsight --configuration runtimeClasspath --dependency io.grpc:grpc-context
*** Skipping the build of codegen and compilation of proto files because skipCodegen=true
  * Skipping the build of Android projects because skipAndroid=true

> Task :grpc-xds:dependencyInsight
Dependency resolution failed because of conflict(s) on the following module(s):
   - io.grpc:grpc-context between versions 1.38.0-SNAPSHOT and 1.27.2

project :grpc-context
   variant "runtimeElements" [
      org.gradle.category            = library
      org.gradle.dependency.bundling = external
      org.gradle.usage               = java-runtime
      org.gradle.libraryelements     = jar
      org.gradle.jvm.version         = 7
   ]
   Selection reasons:
      - By conflict resolution : between versions 1.38.0-SNAPSHOT and 1.27.2

project :grpc-context
+--- project :grpc-api
|    +--- project :grpc-protobuf
|    |    +--- runtimeClasspath
|    |    +--- project :grpc-services
|    |    |    \--- runtimeClasspath
|    |    +--- project :grpc-alts
|    |    |    \--- runtimeClasspath
|    |    \--- project :grpc-grpclb
|    |         \--- project :grpc-alts (*)
|    +--- project :grpc-stub
|    |    +--- runtimeClasspath
|    |    +--- project :grpc-services (*)
|    |    +--- project :grpc-alts (*)
|    |    \--- project :grpc-grpclb (*)
|    +--- project :grpc-core
|    |    +--- runtimeClasspath
|    |    +--- project :grpc-services (*)
|    |    +--- project :grpc-grpclb (*)
|    |    +--- project :grpc-netty
|    |    |    \--- runtimeClasspath
|    |    \--- project :grpc-netty-shaded
|    |         \--- project :grpc-alts (*)
|    +--- project :grpc-auth
|    |    +--- runtimeClasspath
|    |    \--- project :grpc-alts (*)
|    \--- project :grpc-protobuf-lite
|         \--- project :grpc-protobuf (*)
\--- project :grpc-auth (*)

io.grpc:grpc-context:1.27.2 -> project :grpc-context
\--- io.opencensus:opencensus-api:0.28.0
     +--- project :grpc-auth
     |    +--- runtimeClasspath
     |    \--- project :grpc-alts
     |         \--- runtimeClasspath
     +--- com.google.http-client:google-http-client:1.39.1
     |    +--- com.google.auth:google-auth-library-oauth2-http:0.25.2
     |    |    \--- project :grpc-alts (*)
     |    +--- com.google.http-client:google-http-client-gson:1.39.1
     |    |    \--- com.google.auth:google-auth-library-oauth2-http:0.25.2 (*)
     |    \--- com.google.http-client:google-http-client-jackson2:1.39.1
     |         \--- runtimeClasspath
     \--- io.opencensus:opencensus-contrib-http-util:0.28.0
          \--- com.google.http-client:google-http-client:1.39.1 (*)

(*) - dependencies omitted (listed previously)

A web-based, searchable dependency report is available by adding the --scan option.

BUILD SUCCESSFUL in 998ms
1 actionable task: 1 executed

com.google.guava:guava:

suztomo-macbookpro44% ./gradlew :grpc-xds:dependencyInsight --configuration runtimeClasspath --dependency com.google.guava:guava
*** Skipping the build of codegen and compilation of proto files because skipCodegen=true
  * Skipping the build of Android projects because skipAndroid=true

> Task :grpc-xds:dependencyInsight
Dependency resolution failed because of conflict(s) on the following module(s):
   - com.google.guava:guava between versions 30.1-android, 30.0-android and 29.0-android

com.google.guava:guava:30.1-android
   variant "runtime" [
      org.gradle.status              = release (not requested)
      org.gradle.usage               = java-runtime
      org.gradle.libraryelements     = jar
      org.gradle.category            = library

      Requested attributes not found in the selected variant:
         org.gradle.dependency.bundling = external
         org.gradle.jvm.version         = 7
   ]
   Selection reasons:
      - By conflict resolution : between versions 30.1-android, 30.0-android and 29.0-android

com.google.guava:guava:30.1-android
\--- com.google.http-client:google-http-client:1.39.1
     +--- com.google.auth:google-auth-library-oauth2-http:0.25.2
     |    \--- project :grpc-alts
     |         \--- runtimeClasspath
     +--- com.google.http-client:google-http-client-gson:1.39.1
     |    \--- com.google.auth:google-auth-library-oauth2-http:0.25.2 (*)
     \--- com.google.http-client:google-http-client-jackson2:1.39.1
          \--- runtimeClasspath

com.google.guava:guava:29.0-android -> 30.1-android
\--- io.opencensus:opencensus-contrib-http-util:0.28.0
     \--- com.google.http-client:google-http-client:1.39.1
          +--- com.google.auth:google-auth-library-oauth2-http:0.25.2
          |    \--- project :grpc-alts
          |         \--- runtimeClasspath
          +--- com.google.http-client:google-http-client-gson:1.39.1
          |    \--- com.google.auth:google-auth-library-oauth2-http:0.25.2 (*)
          \--- com.google.http-client:google-http-client-jackson2:1.39.1
               \--- runtimeClasspath

com.google.guava:guava:30.0-android -> 30.1-android
+--- project :grpc-alts
|    \--- runtimeClasspath
+--- project :grpc-api
|    +--- project :grpc-protobuf
|    |    +--- runtimeClasspath
|    |    +--- project :grpc-services
|    |    |    \--- runtimeClasspath
|    |    +--- project :grpc-alts (*)
|    |    \--- project :grpc-grpclb
|    |         \--- project :grpc-alts (*)
|    +--- project :grpc-stub
|    |    +--- runtimeClasspath
|    |    +--- project :grpc-services (*)
|    |    +--- project :grpc-alts (*)
|    |    \--- project :grpc-grpclb (*)
|    +--- project :grpc-core
|    |    +--- runtimeClasspath
|    |    +--- project :grpc-services (*)
|    |    +--- project :grpc-grpclb (*)
|    |    +--- project :grpc-netty
|    |    |    \--- runtimeClasspath
|    |    \--- project :grpc-netty-shaded
|    |         \--- project :grpc-alts (*)
|    +--- project :grpc-auth
|    |    +--- runtimeClasspath
|    |    \--- project :grpc-alts (*)
|    \--- project :grpc-protobuf-lite
|         \--- project :grpc-protobuf (*)
+--- project :grpc-auth (*)
+--- project :grpc-core (*)
+--- project :grpc-grpclb (*)
+--- project :grpc-netty (*)
+--- project :grpc-protobuf (*)
+--- project :grpc-protobuf-lite (*)
+--- project :grpc-services (*)
\--- project :grpc-stub (*)

(*) - dependencies omitted (listed previously)

A web-based, searchable dependency report is available by adding the --scan option.

BUILD SUCCESSFUL in 1s
1 actionable task: 1 executed