Only expose interfaces in API

[anuraaga]

We have a use case for completely reimplementing the API - auto instrumentation which does so too provide safety to an app in the face of version skew. Migrating span context in #1935 is helpful, but there are still classes like attributes which can only be bridged with copies. How about we go ahead and expose only interfaces to support this sort of use case better? It does allow a user to implement an interface themselves, but if this actually solves a use case for them, I don't see why we need to proactively block this, and it will be rare.

[bogdandrutu]

From one of the PR that generated this discussion:

@jkwatson:

I've heard this comment a lot, but I've never heard of an actual example of this happening in the real world. Can you show a case where you've been actually impacted by this in the past? Java 8 default methods let us evolve interfaces in a similar way that we've been able to evolve abstract classes, pre Java-8.

@bogdandrutu:

FYI: This is not about evolving the API, but about dealing with third-party implementation of the interface. I am advocating to not allow other implementations for classes when we don't need to offer that capability.

Yes, we had the problem in OpenCensus. We had an interface that represented the Tags (a.k.a baggage) that did not expose publicly a get method, and have our own implementation that exposed (package protected) a get method. Now because that was an interface and everyone can extend the interface it was not trivial to convert to the internal implementation because we had a path where we have to deal with another implementation that does not have our functionality.

You may argue that all functionality must be public, but that is not necessary true if you don't want user to have access to that or abuse your API. By restricting who can implement an interface/abstract class you can rely on package protected functionality to optimize performance etc.

So I am fine with using final classes or auto-value classes with package protected ctor, because it means we control the implementation.

[bogdandrutu]

Also to use @jkwatson expression :), I've heard this comment a lot, that instrumentation needs to re-implement all the classes, but a clear documentation and design document (why is this really required) was never documented in the repository. I saw some documents in google docs (please move that to the repo so others can see and propose better alternatives if possible) which did not convince me that this is a real requirement (mostly like I am missing stuff).

So I suggest that initially we prove that the hypothesis is clear and understood, before taking actions.

[jkwatson]

My opinion on this:

If what you are exposing is pure data, then having a final class is fine. If you're exposing functionality, I will always prefer to have an interface for that functionality.

For example, Baggage is pure, immutable data. Having a final class for that is great. A Span, however, is a combination of data and function, so we should use an interface for it (which we do!).

[bogdandrutu]

@jkwatson same logic applies to SpanContext

[bogdandrutu]

Also there are 2 independent issues that I am hearing:

1. Everything in our API must be extensible, including data only objects.
2. Make all current autovalue generated classes final (using the argument from the guide line). This is wrong because an abstract class with a package protected ctor achieves almost the same behavior as a final class, which is that it cannot be extended by our API users.

[jkwatson]

I don't love AutoValue (mostly because I think the code it generates and requires is not particularly idiomatic), but yes, AutoValue + package private constructors is effectively the same as a final class.

But, given that lombok is such a bad word, AutoValue is the next best thing. And, I'm good with AutoValue data classes, since I'd definitely prefer AutoValue to having to maintain the boilerplate equals/hashCode/toString.

So, my opinion, I think, stands that data classes as final (or AutoValue) is good. Classes with functionality should be interfaces, rather than classes.

[anuraaga]

@trask can elaborate on the agent's shading constraints. I think the short answer is that agents have broken user apps in the past due to version issues and the shading of the API is important to maintain the agent's safety guarantees. It'd definitely be good to get that into markdown.

@tylerbenson also has a use case for extending spancontext it seems and maybe can explain it.

There is some double edged sword to interface vs class, but I think we can always come up with examples of both helpful and harmful. Another fun thing to do with SpanContext as an interface would be to have the Span implementations implement it and inline the data to reduce an allocation. It's impossible to make such changes in the future with classes.

I think it's fair to assume that the burden of something working is on the implementer if they do something custom. It's possible to document this, and point at it if something goes wrong with something custom.

We can also look at our current API to see what could break with something custom. I don't think we have anything - this is probably because the fact that the API needs to be used by SDKs tends to prevent package private tricks.

So I would still consider more interfaces, precisely to have more flexibility in the future, not less. I think this is possible and reasonable precisely because we have such a strict separation between API and SDK.

[bogdandrutu]

I think changing the implementation for a final class is backwards compatible. Maybe this is yet a missing defined thing, what is considered to be a breaking change and what is not after the GA.

[bogdandrutu]

@anuraaga i would really like to see the requirement of having to re-implement all classes documented and explain, would make a lot of the discussions easier if we agree on that.

[anuraaga]

I think changing the implementation for a final class is backwards compatible.

Yeah, but it's not possible to, for example, inline it into a different class unless it's an interface. A class can't be changed to an interface and preserve ABI.

[bogdandrutu]

Yet another concept that is not documented, do we really need to guarantee ABI compatibility?

PS: I am trying to identity all the missing requirements that some of us have in their minds and are not documented and clarify between devs. I would say that we definitely need to document these before making a decision, so that we can have all pros/cons.

[trask]

do we really need to guarantee ABI compatibility?

I would think yes, due to libraries depending on otel api, which won't get recompiled when application developer bumps otel api version.

PS: I am trying to identity all the missing requirements that some of us have in their minds and are not documented and clarify between devs

💯👍