Skip to content

Convert between a Table and a binary format #638

@lars-reimann

Description

@lars-reimann

Is your feature request related to a problem?

Sending a Table between the runner and the DSL is currently quite expensive, since we convert the table to a dictionary and parse that again.

Desired solution

Investigate whether a compressible binary format like parquet or ORC would speed this up. Choose one format and add to_..._file and from_..._file methods for it. Maybe also add a version that returns bytes instead of writing to a file.

Possible alternatives (optional)

No response

Screenshots (optional)

No response

Additional Context (optional)

No response

Metadata

Metadata

Assignees

Labels

releasedIncluded in a release

Type

No type

Projects

Status

✔️ Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions