-
Notifications
You must be signed in to change notification settings - Fork 4k
Closed as not planned
Labels
Component: RStatus: stale-warningIssues and PRs flagged as stale which are due to be closed if no indication otherwiseIssues and PRs flagged as stale which are due to be closed if no indication otherwiseType: enhancement
Description
There's a bit of technical debt accumulated in array_to_vector and vector_to_array:
- Mix of conversion and casting, ideally we'd move casting out of there (at the cost of more memory copy). The rationale is that the conversion logic will differ from the CastKernels, e.g. when to raise errors, benefits from complex conversions like timezone... The current implementation is fast, e.g. it fuses the conversion and casting in a single loop at the cost of code clarity and divergence.
- There should be 2 paths, zero-copy, non zero-copy. The non-zero copy should use the newly introduced VectorToArrayConverter which will work with complex nested types.
- The in array_to vector, Converter should work primarily with Array and not ArrayVector
- The vector_to_array should not use builders, sizes are known, the null bitmap should be constructed separately. There's probably a chance that we can re-use R's memory with zero-copy for the raw data.
- There seem to be multiple paths that do the same conversion: ARROW-6235: [R] Implement conversion from arrow::BinaryArray to R character vector #7514 (comment)
Reporter: Francois Saint-Jacques / @fsaintjacques
Related issues:
- [R] Zero-copy Arrow to R where possible (relates to)
- [R] Use Converter API to convert SEXP to Array/ChunkedArray (relates to)
- [R] Table$create with schema crashes with some dictionary index types (is related to)
- [R] Support creating ListArray from R list (is related to)
Note: This issue was originally created as ARROW-7798. Please see the migration documentation for further details.
Metadata
Metadata
Assignees
Labels
Component: RStatus: stale-warningIssues and PRs flagged as stale which are due to be closed if no indication otherwiseIssues and PRs flagged as stale which are due to be closed if no indication otherwiseType: enhancement