Support key_based batch builder for functions and sources

[wolfstudy]

Signed-off-by: xiaolong.ran rxl@apache.org

Motivation

Currently, we support the Key_Shared subscription mode in Pulsar Function. In order to ensure that when batch is turned on, we can also ensure that messages are distributed to different consumers in the correct order, so we need to support the batch builder of KEY_BASED in Pulsar Functions.

Modifications

• Add --batch-builder for Pulsar Functions
• Add --batch-builder for Pulsar Sources
• Add test case

[wolfstudy]

/pulsarbot run-failure-checks

[sijie]

Overall looks good. However, I didn't see any test case. Can you add one?

[wolfstudy]

Overall looks good. However, I didn't see any test case. Can you add one?

The current unit tests can be covered. The change of this pull request is to add a batchBuilder field to ProducerConfig. The current unit test framework covers the Pojo object ProducerConfig, so we only need to add this field to the existing unit test framework. Refer to pulsar-functions/utils/src/test/java/org/apache/pulsar/functions/utils/SourceConfigUtilsTest.java and pulsar-functions/utils/src/test/java/org/apache/pulsar/functions/utils/FunctionConfigUtilsTest.java.

[wolfstudy]

You mean, do we add another integration test on top of the unit test?

[wolfstudy]

@sijie PTAL again thanks. I added the --batch-builder parameter to the existing integration test to confirm whether this parameter is correctly identified.