-
Notifications
You must be signed in to change notification settings - Fork 4k
Closed
Description
The IPC reader operates at the granularity of an entire record batch; even if you're loading only a few columns, the entire record batch is read. ARROW-12683 partially addressed this issue by enabling this capability for sync read API, however, async read API still lacks of this capability.
Reporter: Yue Ni / @niyue
Assignee: Weston Pace / @westonpace
Related issues:
- [C++] RecordBatchFileReader performance really bad in S3 (is duplicated by)
- [C++] Enable fine-grained I/O (coalescing) in IPC reader (relates to)
PRs and other links:
Note: This issue was originally created as ARROW-14577. Please see the migration documentation for further details.