Skip to content

Parallelise chunk fetches #599

@bboreham

Description

@bboreham

At least for the DynamoDB back-end, chunks are fetched in batches of 100, serially.

The 100 limit comes from Amazon.

Brief experimentation shows it will go way faster if we parallelise the fetches.

For extra credit, split by table, by hash-key, etc., to maximise use of available parallelisation inside DynamoDB.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions