-
Notifications
You must be signed in to change notification settings - Fork 4k
Closed as not planned
Labels
Component: PythonStatus: stale-warningIssues and PRs flagged as stale which are due to be closed if no indication otherwiseIssues and PRs flagged as stale which are due to be closed if no indication otherwiseType: enhancement
Description
When converting python objects to arrow array currently we always collect the input to a sequence, but this may be memory consuming in certain cases.
For unknown sized iterators we could consume and temporarily store the seen items during inference potentially improving both the conversion time and peak memory usage.
Reporter: Krisztian Szucs / @kszucs
Related issues:
Note: This issue was originally created as ARROW-10006. Please see the migration documentation for further details.
Metadata
Metadata
Assignees
Labels
Component: PythonStatus: stale-warningIssues and PRs flagged as stale which are due to be closed if no indication otherwiseIssues and PRs flagged as stale which are due to be closed if no indication otherwiseType: enhancement