Skip to content

Prototype delayed dask dataframes for better computation scaling #9

@evamaxfield

Description

@evamaxfield

Currently all computation is multithreaded by default and that doesn't leave a lot of room to the user in terms of deciding how and when they want to actually gather data or compute a result.

The datasets module can easily be switched over to dask for data gathering and caching

The keywords module should be looked at for how to use dask dataframes for just-in-time compute and out-of-memory compute.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions