Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions r/DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ Imports:
utils,
vctrs
Roxygen: list(markdown = TRUE, r6 = FALSE, load = "source")
RoxygenNote: 7.1.2
RoxygenNote: 7.2.0
Config/testthat/edition: 3
VignetteBuilder: knitr
Suggests:
Expand Down Expand Up @@ -88,7 +88,6 @@ Collate:
'dataset-partition.R'
'dataset-scan.R'
'dataset-write.R'
'deprecated.R'
'dictionary.R'
'dplyr-arrange.R'
'dplyr-collect.R'
Expand Down
4 changes: 2 additions & 2 deletions r/NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,7 @@ export(FileType)
export(FixedSizeListArray)
export(FixedSizeListType)
export(FragmentScanOptions)
export(GcsFileSystem)
export(HivePartitioning)
export(HivePartitioningFactory)
export(InMemoryDataset)
Expand Down Expand Up @@ -251,6 +252,7 @@ export(arrow_available)
export(arrow_info)
export(arrow_table)
export(arrow_with_dataset)
export(arrow_with_gcs)
export(arrow_with_json)
export(arrow_with_parquet)
export(arrow_with_s3)
Expand Down Expand Up @@ -330,7 +332,6 @@ export(null)
export(num_range)
export(one_of)
export(open_dataset)
export(read_arrow)
export(read_csv_arrow)
export(read_delim_arrow)
export(read_feather)
Expand Down Expand Up @@ -366,7 +367,6 @@ export(utf8)
export(value_counts)
export(vctrs_extension_array)
export(vctrs_extension_type)
export(write_arrow)
export(write_csv_arrow)
export(write_dataset)
export(write_feather)
Expand Down
11 changes: 6 additions & 5 deletions r/NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
* `lubridate::parse_date_time()` datetime parser:
* `orders` with year, month, day, hours, minutes, and seconds components are supported.
* the `orders` argument in the Arrow binding works as follows: `orders` are transformed into `formats` which subsequently get applied in turn. There is no `select_formats` parameter and no inference takes place (like is the case in `lubridate::parse_date_time()`).
* `read_arrow()` and `write_arrow()`, deprecated since 1.0.0 (July 2020), have been removed. Use the `read/write_feather()` and `read/write_ipc_stream()` functions depending on whether you're working with the Arrow IPC file or stream format, respectively.

# arrow 8.0.0

Expand Down Expand Up @@ -50,7 +51,7 @@

## Enhancements to date and time support

* `read_csv_arrow()`'s readr-style type `T` is mapped to `timestamp(unit = "ns")`
* `read_csv_arrow()`'s readr-style type `T` is mapped to `timestamp(unit = "ns")`
instead of `timestamp(unit = "s")`.
* For Arrow dplyr queries, added additional `{lubridate}` features and fixes:
* New component extraction functions:
Expand Down Expand Up @@ -86,14 +87,14 @@
record batches, arrays, chunked arrays, record batch readers, schemas, and
data types. This allows other packages to define custom conversions from their
types to Arrow objects, including extension arrays.
* Custom [extension types and arrays](https://arrow.apache.org/docs/format/Columnar.html#extension-types)
* Custom [extension types and arrays](https://arrow.apache.org/docs/format/Columnar.html#extension-types)
can be created and registered, allowing other packages to
define their own array types. Extension arrays wrap regular Arrow array types and
provide customized behavior and/or storage. See description and an example with
`?new_extension_type`.
* Implemented a generic extension type and as_arrow_array() methods for all objects where
`vctrs::vec_is()` returns TRUE (i.e., any object that can be used as a column in a
`tibble::tibble()`), provided that the underlying `vctrs::vec_data()` can be converted
* Implemented a generic extension type and as_arrow_array() methods for all objects where
`vctrs::vec_is()` returns TRUE (i.e., any object that can be used as a column in a
`tibble::tibble()`), provided that the underlying `vctrs::vec_data()` can be converted
to an Arrow Array.

## Concatenation Support
Expand Down
18 changes: 0 additions & 18 deletions r/R/dataset-scan.R
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,6 @@
#' * `filter`: A `Expression` to filter the scanned rows by, or `TRUE` (default)
#' to keep all rows.
#' * `use_threads`: logical: should scanning use multithreading? Default `TRUE`
#' * `use_async`: logical: deprecated, this field no longer has any effect on
#' behavior.
#' * `...`: Additional arguments, currently ignored
#' @section Methods:
#' `ScannerBuilder` has the following methods:
Expand All @@ -45,7 +43,6 @@
#' - `$UseThreads(threads)`: logical: should the scan use multithreading?
#' The method's default input is `TRUE`, but you must call the method to enable
#' multithreading because the scanner default is `FALSE`.
#' - `$UseAsync(use_async)`: logical: deprecated, has no effect
#' - `$BatchSize(batch_size)`: integer: Maximum row count of scanned record
#' batches, default is 32K. If scanned record batches are overflowing memory
#' then this method can be called to reduce their size.
Expand Down Expand Up @@ -73,19 +70,11 @@ Scanner$create <- function(dataset,
projection = NULL,
filter = TRUE,
use_threads = option_use_threads(),
use_async = NULL,
batch_size = NULL,
fragment_scan_options = NULL,
...) {
stop_if_no_datasets()

if (!is.null(use_async)) {
.Deprecated(msg = paste(
"The parameter 'use_async' is deprecated",
"and will be removed in a future release."
))
}

if (inherits(dataset, "arrow_dplyr_query")) {
if (is_collapsed(dataset)) {
# TODO: Is there a way to get a RecordBatchReader rather than evaluating?
Expand Down Expand Up @@ -258,13 +247,6 @@ ScannerBuilder <- R6Class("ScannerBuilder",
dataset___ScannerBuilder__UseThreads(self, threads)
self
},
UseAsync = function(use_async = TRUE) {
.Deprecated(msg = paste(
"The function 'UseAsync' is deprecated and",
"will be removed in a future release."
))
self
},
BatchSize = function(batch_size) {
dataset___ScannerBuilder__BatchSize(self, batch_size)
self
Expand Down
40 changes: 0 additions & 40 deletions r/R/deprecated.R

This file was deleted.

2 changes: 1 addition & 1 deletion r/R/dplyr-union.R
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ union.arrow_dplyr_query <- function(x, y, ...) {
x <- as_adq(x)
y <- as_adq(y)

distinct(union_all(x, y))
dplyr::distinct(dplyr::union_all(x, y))
}

union.Dataset <- union.ArrowTabular <- union.RecordBatchReader <- union.arrow_dplyr_query
Expand Down
6 changes: 4 additions & 2 deletions r/man/ArrayData.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions r/man/FileSystem.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 4 additions & 2 deletions r/man/Scalar.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 0 additions & 3 deletions r/man/Scanner.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 4 additions & 2 deletions r/man/array.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion r/man/arrow-package.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions r/man/arrow_info.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

11 changes: 4 additions & 7 deletions r/man/read_ipc_stream.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 2 additions & 5 deletions r/man/write_ipc_stream.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

53 changes: 10 additions & 43 deletions r/tests/testthat/test-Table.R
Original file line number Diff line number Diff line change
Expand Up @@ -15,49 +15,6 @@
# specific language governing permissions and limitations
# under the License.

# Common fixtures used in many tests
tbl <- tibble::tibble(
int = 1:10,
dbl = as.numeric(1:10),
lgl = sample(c(TRUE, FALSE, NA), 10, replace = TRUE),
chr = letters[1:10],
fct = factor(letters[1:10])
)
tab <- Table$create(tbl)

test_that("read_table handles various input streams (ARROW-3450, ARROW-3505)", {
tbl <- tibble::tibble(
int = 1:10, dbl = as.numeric(1:10),
lgl = sample(c(TRUE, FALSE, NA), 10, replace = TRUE),
chr = letters[1:10]
)
tab <- Table$create(!!!tbl)

tf <- tempfile()
on.exit(unlink(tf))
expect_deprecated(
write_arrow(tab, tf),
"write_feather"
)

tab1 <- read_feather(tf, as_data_frame = FALSE)
tab2 <- read_feather(normalizePath(tf), as_data_frame = FALSE)

readable_file <- ReadableFile$create(tf)
expect_deprecated(
tab3 <- read_arrow(readable_file, as_data_frame = FALSE),
"read_feather"
)
readable_file$close()

mmap_file <- mmap_open(tf)
mmap_file$close()

expect_equal(tab, tab1)
expect_equal(tab, tab2)
expect_equal(tab, tab3)
})

test_that("Table cast (ARROW-3741)", {
tab <- Table$create(x = 1:10, y = 1:10)

Expand Down Expand Up @@ -98,6 +55,16 @@ test_that("Table $column and $field", {
expect_error(tab$field("one"))
})

# Common fixtures used in some of the following tests
tbl <- tibble::tibble(
int = 1:10,
dbl = as.numeric(1:10),
lgl = sample(c(TRUE, FALSE, NA), 10, replace = TRUE),
chr = letters[1:10],
fct = factor(letters[1:10])
)
tab <- Table$create(tbl)

test_that("[, [[, $ for Table", {
expect_identical(names(tab), names(tbl))

Expand Down
Loading