-
Notifications
You must be signed in to change notification settings - Fork 4k
Closed
Description
In ARROW-14908 we solved the case of joining a dataset to an in memory table, but did not solve joining two datasets.
The previous solution was to add +1 to the thread count, because the hash join logic might be called by the scanner's IO thread. For joining more than 1 dataset, we might have more than 1 IO thread, so we either need to add a larger arbitrary number or find a way to make the state logic more resilient to unexpected threads.
Reporter: Will Jones / @wjones127
Assignee: Will Jones / @wjones127
Related issues:
- [R] join on dataset crashes on Windows (is related to)
PRs and other links:
Note: This issue was originally created as ARROW-15718. Please see the migration documentation for further details.