Skip to content

Add custom bulk processor#153

Merged
HuiJun merged 3 commits intorelease/4.0.2from
feature/bulkprocessing
Feb 18, 2021
Merged

Add custom bulk processor#153
HuiJun merged 3 commits intorelease/4.0.2from
feature/bulkprocessing

Conversation

@HuiJun
Copy link
Collaborator

@HuiJun HuiJun commented Feb 18, 2021

No description provided.

}

public void clear() {
if (allRequests.size() > bulkLimit) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'>='

}

public void close() {
batches(allRequests, bulkLimit).forEach(this::bulkBatchRequests);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you still need to batch? since adds always clear it when it reaches the limit anything left shouldn't go above the limit

protected int bulkLimit;
RestHighLevelClient client;
RequestOptions options;
List<DocWriteRequest<?>> allRequests = new ArrayList<>();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can just be BulkRequest, then bulkBatchRequests don't need to readd them all when sending

BulkRequest bulkRequest = new BulkRequest();
actionRequest.forEach(bulkRequest::add);
try {
client.bulk(bulkRequest, options);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need to check result for failures and throw stuff

@HuiJun HuiJun requested a review from dlamoris February 18, 2021 01:19
@HuiJun HuiJun merged commit df22ea2 into release/4.0.2 Feb 18, 2021
@HuiJun HuiJun deleted the feature/bulkprocessing branch February 18, 2021 01:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants