Why is this needed?
Historically, the Batch utility supported SQS event sources. Today, it supports more than just SQS. It also supports Kinesis and DynamoDB stream event sources.
These additionally event sources were added as a separate "handlers" in:
While this approach works, it also introduces a lot of code duplication since the whole code is repeated for different method overloads such as for processBatchInParallel and also in between the different handlers.
This also leads regularly to Sonarcube findings when sending PRs involving these handlers.
Which area does this relate to?
Batch processing
Suggestion
This issue suggest to refactor the batch module in a way to remove this redundant logic since the majority of code is shared. We should find a way to separate the orchestration logic of the (parallel) batch processing from the event source specific responses.
Anyone interested in working on this, please feel free to propose refactoring suggestions.
Acknowledgment
Why is this needed?
Historically, the Batch utility supported SQS event sources. Today, it supports more than just SQS. It also supports Kinesis and DynamoDB stream event sources.
These additionally event sources were added as a separate "handlers" in:
While this approach works, it also introduces a lot of code duplication since the whole code is repeated for different method overloads such as for
processBatchInParalleland also in between the different handlers.This also leads regularly to Sonarcube findings when sending PRs involving these handlers.
Which area does this relate to?
Batch processing
Suggestion
This issue suggest to refactor the batch module in a way to remove this redundant logic since the majority of code is shared. We should find a way to separate the orchestration logic of the (parallel) batch processing from the event source specific responses.
Anyone interested in working on this, please feel free to propose refactoring suggestions.
Acknowledgment