Remove GSON from our public API

[QuintinWillison]

Reported by a customer (internal conversation). In their words, given this tripped them up...

Initial Report:

class HttpPaginatedResponse, returned by AblyRest.request method returns an array of com.google.gson.JsonElement from its items() method, yet the ably-java library (v1.1.10) declares only a runtime dependency on GSON library, leading my IDE to flag calls to the items() method as an error with message ‘Cannot access com.google.gson.JsonElement’. Interested to know why GSON is not a compile-time dependency given its classes are used in the public API of ably-java?

Follow-up opinion as to how this should be instead:

implementation results in those dependencies being declared as ‘runtime’ dependencies in the published POM, hence they are transitively included in the consuming applications runtimeClasspath but not the compileClasspath. Personally, I would consider having GSON as an api dependency in the project (this would also require use of the Gradle ‘java-library’ plugin rather than the plain ‘java’ plugin), meaning it would end up as ‘compile’ dependency in the ably-java POM and thus be available in the compileClasspath of consuming applications. I suspect what may have happened is that when the original Gradle compile scope (which resulted in dependencies being published as ‘compile’ scope in the POM) was deprecated, someone simply replaced it with implementation (as I originally did in all my personal projects!) without fully appreciating the subtle difference between implementation and api.

Like most things, no right or wrong (there are downsides to using api), but maybe worth thinking about. https://docs.gradle.org/current/userguide/java_library_plugin.html

┆Issue is synchronized with this Jira Research by Unito

[QuintinWillison]

I've reproduced this in IntelliJ IDEA CE when working with a brand new Gradle project. I have the ably-java Java jar in a libs folder and introduced it to the project using:

dependencies {
    implementation fileTree(include: ['*.jar'], dir: 'libs')
}

It builds fine:

hello-deltas-java % ./gradlew build

BUILD SUCCESSFUL in 638ms
5 actionable tasks: 4 executed, 1 up-to-date

but crashes at runtime:

hello-deltas-java % ./gradlew run

> Task :run FAILED
Ably Client Library: java v.1.2.0
Exception in thread "main" java.lang.NoClassDefFoundError: com/google/gson/JsonElement
        at io.ably.lib.rest.Auth.setTokenDetails(Auth.java:928)
        at io.ably.lib.rest.Auth.<init>(Auth.java:893)
        at io.ably.lib.rest.AblyBase.<init>(AblyBase.java:72)
        at io.ably.lib.rest.AblyRest.<init>(AblyRest.java:25)
        at io.ably.lib.realtime.AblyRealtime.<init>(AblyRealtime.java:53)
        at io.ably.lib.realtime.AblyRealtime.<init>(AblyRealtime.java:44)
        at io.ably.examples.deltas.Hello.main(Hello.java:11)
Caused by: java.lang.ClassNotFoundException: com.google.gson.JsonElement
        at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
        ... 7 more

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':run'.
> Process 'command '/Library/Java/JavaVirtualMachines/jdk1.8.0_241.jdk/Contents/Home/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 547ms
2 actionable tasks: 1 executed, 1 up-to-date

The code doesn't do anything valuable yet - all I have so far is:

package io.ably.examples.deltas;

import io.ably.lib.BuildConfig;
import io.ably.lib.realtime.AblyRealtime;
import io.ably.lib.types.AblyException;

public class Hello {
    public static void main(final String[] args) throws AblyException {
        System.out.println("Ably Client Library: " + BuildConfig.NAME + " v." + BuildConfig.VERSION);

        final AblyRealtime ably = new AblyRealtime("xxxxx");

    }
}

[QuintinWillison]

Actually that could be unrelated, being a direct result of me trying to 'manually' bring this in as a jar. I've added GSON dependency but now getting:

java.lang.ClassNotFoundException: org.java_websocket.client.WebSocketClient

[paddybyers]

But adding it as a dependency in the way that you have isn't going to work because you're not bringing in any of the library dependencies. That's why the README says to reference it via

compile 'io.ably:ably-java:1.1.11'

Otherwise you'd need a fat jar, and we don't publish a fat jar. I don't think what you're seeing is an issue specific to gson.

[QuintinWillison]

We bring GSON in as an implementation dependency, but we expose it as part of our API - see MessageExtras' constructor, for example.

I expect that we either need to change the dependency from implementation to api (simple fix) or consider making a breaking change so that Gson APIs like JsonElement and JsonObject are not exposed directly via our API (more predictable fix for supportability going forwards).

[paddybyers]

I expect that we either need to change the dependency from implementation to api (simple fix)

My understanding is that this change won't work. It will enable our code to compile against the API declared by that dependency, but it won't cause the dependency to be included in any build that includes our library. Then the dependency will be missing at runtime.

consider making a breaking change so that Gson APIs like JsonElement and JsonObject are not exposed directly via our API (more predictable fix for supportability going forwards)

I'm interested to see what that would be.

The ideal thing would be that the platforms (Android and JDK) have a de facto standard JSON serialisability interface, at least, and then we wouldn't need to make that decision ourselves.

[lmars]

Just to add an example of this problem, when developing the Kafka Connect Ably Connector I needed to set message extras which involved importing io.ably.lib.util.JsonUtils.JsonUtilsObject, and that failed to compile unless I explicitly added the gson dependency. My Java experience is limited though, so I may well have done that incorrectly.

[KacperKluka]

I've encountered this issue in Ably Asset Tracking too. When I was parsing the presence data sent from ably-js client it turned out that it comes in the JsonObject format which is a type from the Gson library. Luckily, we're also using the Gson library in the project but there are other popular JSON parsing libraries (like Moshi) that other people might be using in their projects.
If we could somehow remove the Gson dependency from the API it would be awesome.