ARROW-11766: [R] Better handling for missing compression codecs on Linux #9880
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Member
|
I suspect the CI failures are related to what was resolved in #9868, so you could try to rebase to get rid of those. I also see a number of comments from the now closed #9743 that haven't yet been addressed in this branch/PR would you mind looking at those, addressing them and letting us know when this is ready for re-review? |
Contributor
Author
thanks, I shall write a list here |
Contributor
Author
|
List of TODOs:
|
…run synchronously from datasets Calling the async streaming CSV reader from the synchronous Scanner::Scan was causing a form of nested parallelism and causing nested deadlocks. This commit brings over some of the work in ARROW-7001 and allows the CSV scan task to be called in an async fashion. In addition, an async path is put in the scanner and dataset write so that all internal uses of ScanTask()->Execute happen in an async-friendly way. External uses of ScanTask()->Execute should already be outside the CPU thread pool and should not cause deadlock. Some of this PR will be obsoleted by ARROW-7001 but the work in file_csv and the test cases should remain fairly intact. Closes apache#9868 from westonpace/bugfix/arrow-12161 Lead-authored-by: Weston Pace <weston.pace@gmail.com> Co-authored-by: Antoine Pitrou <antoine@python.org> Signed-off-by: David Li <li.davidm96@gmail.com>
… integration tests # Rationale Rust debug symbols are quite verbose, taking up memory during the final link time as well as significant disk space. Turning off the creation of symbols should save us compile / test time for CI as well as space on the integration test # Change Do not produce debug symbols on Rust CI (keep enough to have line numbers in `panic!` traceback, but not enough to interpret a core file, which no one does to my knowledge anyways) Note that the integration test passed: https://github.com/apache/arrow/pull/9879/checks?check_run_id=2256148363 Closes apache#9879 from alamb/less_symbols_in_integration Authored-by: Andrew Lamb <andrew@nerdnetworks.org> Signed-off-by: Andrew Lamb <andrew@nerdnetworks.org>
This updates zstd version used by parquet crate to zstd = "0.7.0+zstd.1.4.9". Closes apache#9881 from aldanor/feature/zstd-0.7 Authored-by: Ivan Smirnov <i.s.smirnov@gmail.com> Signed-off-by: Andrew Lamb <andrew@nerdnetworks.org>
This just moves the tests to allow the feature-flag to be used and pass this kind of test (where previously it would fail) ```bash cargo test --no-default-features --features cli ``` Closes apache#9874 from seddonm1/regexp_match_test Authored-by: Mike Seddon <seddonm1@gmail.com> Signed-off-by: Andrew Lamb <andrew@nerdnetworks.org>
Closes apache#9763 from emkornfield/trivial_prs Lead-authored-by: Micah Kornfield <emkornfield@gmail.com> Co-authored-by: emkornfield <micahk@google.com> Signed-off-by: Micah Kornfield <emkornfield@gmail.com>
This depends on ARROW-12192: apache/arrow-site#99 Closes apache#9885 from kou/release-post-website-download Lead-authored-by: Sutou Kouhei <kou@clear-code.com> Co-authored-by: Sutou Kouhei <kou@cozmixng.org> Signed-off-by: Neal Richardson <neal.p.richardson@gmail.com>
Contributor
Author
|
gonna need new PR :S |
github-actions
bot
added
Component: Rust - DataFusion
Component: C++
Component: Rust
labels
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.