-
Notifications
You must be signed in to change notification settings - Fork 4k
Closed
Description
In ARROW-11843 we found that we still can't enable parallel column conversion for Parquet datasets because the Arrow Parquet reader uses OptionalParallelFor, which does a parallel wait. We should provide an async version to avoid nested parallelism.
Reporter: David Li / @lidavidm
Assignee: David Li / @lidavidm
Related issues:
- [C++][Parquet] Always transfer to executor in Parquet AsyncGenerator (is duplicated by)
- [C++] Add asynchronous read to parquet::arrow::FileReader (relates to)
PRs and other links:
Note: This issue was originally created as ARROW-12597. Please see the migration documentation for further details.