#4815 introduced the incremental handoff for KafkaIndexTask. In this PR, publishExecService was added, but it breaks the below contract of Appenderator.
If committer is not provided, no metadata is persisted. If it's provided, the add, clear, persist, persistAll, and push methods should all be called from the same thread to keep the metadata committed by Committer in sync.
One solution is adding synchronization blocks to KafkaIndexTask when it calls AppenderatorDriver.add(), AppenderatorDriver.persist(), and AppenderatorDriver.publish(). However, I'm not sure why publishExecService is needed because Appenderator and AppenderatorDriver are already supporting background publishing.
#4815 introduced the incremental handoff for KafkaIndexTask. In this PR, publishExecService was added, but it breaks the below contract of
Appenderator.One solution is adding synchronization blocks to KafkaIndexTask when it calls
AppenderatorDriver.add(),AppenderatorDriver.persist(), andAppenderatorDriver.publish(). However, I'm not sure whypublishExecServiceis needed becauseAppenderatorandAppenderatorDriverare already supporting background publishing.