Background
The goal is to prevent Hyperdrive job instance creations when there are no messages available on a kafka topic (or only a few). To that end, the offsets from the checkpoint folder should be compared with the latest offsets from the kafka topic.
This issue is about getting the offsets from the kafka topic.
All configurations (including keytab location, broker urls, certificate locations) needed to access the kafka topic should be available from the resolved job definition.
Task
A service method should be written, that takes a ResolvedJobDefinition and returns the latest offset from the kafka topic.
Related
#687, #690