KAFKA-13328, KAFKA-13329 (1): Add preflight validations for key, value, and header converter classes

[C0urante]

Jira 1, Jira 2

Depends on #14303

Adds preflight validation checks for key, value, and header converter classes specified in connector configurations. For each converter type, these checks verify that:

• The class has a public, no-args constructor
• The class can be instantiated without an exception being thrown
• The class is not abstract
• • In this case, a list of possible child classes is provided in a user-friendly error message
• The class implements the expected interface

There are two components to the linked Jira tickets: this validation (which is fairly straightforward), and more sophisticated logic that leverages ConfigDef objects provided by these plugin classes. For ease of review, I'm planning on filing two PRs for these tickets, but instead of grouping by ticket (which would include all key/value converter changes in one ticket and all header converter changes in another), I think it might be easier to group by the kind of validation we're performing. If that's acceptable, the next PR will add ConfigDef-based validation for key, value, and header converters.

Committer Checklist (excluded from commit message)

• Verify design and implementation
• Verify test coverage and CI build status
• Verify documentation (including upgrade notes)

[C0urante]

This is overkill for a single use (which is the case in this PR), but a second use case is added for it in #14309.

[gharris1727]

I found more use-cases for this today, so 👍 on adding this helper.

[yashmayya]

Thanks Chris, this is another really nice improvement! I just had a few comments and questions.

[gharris1727]

We have all of the checks already, why didn't we add the validators! 🙃

Thanks for this, Chris.

[gharris1727]

The first argument to this exception should be the config name, not the type name like it's used in the exception message and in the call-sites.

[C0urante]

🤦 Thanks, good catch.

In pursuit of simplicity I've taken a stab at using the single-arg ConfigDef constructor, and using a one-size-fits-all error message that just says "This class" instead of, e.g., "Transform". Since all current uses for this code path involve (directly or indirectly) hitting a ConfigDef.Validator as part of config validation, I think the user-facing impact should only go as far as the small wording change. Let me know if you think it's better to preserve existing behavior, though.

[gharris1727]

I like this simplification, and it makes sense to generalize the error message as part of promoting this to the Utils class. parseForValidate will still attribute the error to the correct key/value.

[gharris1727]

If AbstractClass.Key implemented the base interface, but didn't extend AbstractClass, this wouldn't show up in this listing. If this was baseClass::isAssignableFrom, it would find classes like that.

I don't know how often people would create Key and Value inner classes without subclassing the outer class, since that seems to be the reason people use the inner classes in the first place. But if you change this to take the baseClass and check baseClass.isAssignableFrom(cls) like both of the call-sites, you could change this filter statement too. I don't think this is a big deal, so feel free to leave this implementation as-is.

[C0urante]

I took a stab at something that would accept a base class and relax the filtering logic here accordingly, but it didn't quite feel right since we don't do verification that cls extends baseClass. I guess we could add that check as well and have something like Utils::ensureConcreteSubclass--how does that sound? I'm on the fence about it potentially doing too much for a utility method, but it does provide user-facing advantages.

[gharris1727]

I guess we could add that check as well and have something like Utils::ensureConcreteSubclass--how does that sound?

Yeah I imagine a ensureConcreteSubclass(Class<?> cls, Class<?> baseClass) that:

1. asserts that cls is concrete and baseClass.isAssignableFrom(cls) is true
2. if not valid, suggests child classes that are concrete andbaseClass::isAssignableFrom is true.

I don't think that it would compose well with the current ensureConcrete method though, so it would probably replace what is there already.

I'm on the fence about it potentially doing too much for a utility method, but it does provide user-facing advantages.

I thought that about the child class suggestion logic, but relented for the same reason. It significantly complicates this method which could otherwise be a single if condition, but that complication could help someone quickly resolve an obvious typo.

I don't think that asserting baseClass.isAssignableFrom(cls) is too much for this utility method, because we should always have a baseClass in mind when defining a CLASS configuration. Users of the AbstractConfig::getConfiguredInstance methods already need to provide a baseClass for subclass checking and type safety at runtime, this utility method would help users to remember the same check at validation time.

And worst-case, someone can just specify Object as the baseClass, and effectively disable the baseClass filtering.

[C0urante]

@gharris1727 sorry for the delay, know it's been a minute! I've implemented your suggestions with the latest commit. This should be ready for another round whenever you have time.

[gharris1727]

LGTM!

[C0urante]

Thanks Greg!