data.table(double(0), 1) returns a zero-row table

[shrektan]

One of pattern of using data.table is to return a data.table at the end of a function, like:

data.table(
  A = xxx,
  B = yyy,
  Z = zzz,
)

I often need to include some constant columns, like the datetime, the id, etc. Most of time it returns what I want except when all other inputs are zero length vector. For example:

data.table::data.table(
  a = double(),
  b = 1
)
#> Warning in as.data.table.list(x, keep.rownames = keep.rownames, check.names
#> = check.names): Item 1 has 0 rows but longest item has 1; filled with NA
#>     a b
#> 1: NA 1

So I have to explicitly use rep() everytime in order to ensure it returns the desired result.

a <- double()
n <- length(a)
data.table::data.table(
  a = a,
  b = rep(a, n)
)
#> Empty data.table (0 rows and 2 cols): a,b

It's verbose, inconvenient and often being forgotten in the first place because in my mind, the length 1 column should always be recycled... so it's actually a little bit inconsistent, IMO:

### gets a data.table with row 2
data.table::data.table(
  a = double(2),
  b = 1
)
### gets a data.table with row 3
data.table::data.table(
  a = double(3),
  b = 1
)
### Should get a zero-row data.table but gets a 1-row table with a warning
data.table::data.table(
  a = double(0),
  b = 1
)

while tibble gets what I expect:

tibble::tibble(
  a = double(0),
  b = 1
)
#> # A tibble: 0 x 2
#> # ... with 2 variables: a <dbl>, b <dbl>

[MichaelChirico]

Related: #2662 and #759

[shrektan]

This is very annoying, IMO. Another case is in querying with J(). In the below example, I expect a zero vector returned. Imagine the program has many places that use this kind of query and one day there's a zero-length input that occurred, the further execution became a mess - as it doesn't guarantee the length of the output equals the input.

dt <- data.table::data.table(CODE = c('a', 'b'), DATE = 1:2, VALUE = c(1.3, 1.5),
                             key = c('CODE', 'DATE'))
dt[J('a', 1), VALUE]
#> [1] 1.3
dt[J(character(), 1), VALUE]
#> Warning in as.data.table.list(i): Item 1 has 0 rows but longest item has 1;
#> filled with NA
#> [1] NA ##### but I expect a double()

As we've already provided a warning here, the chance that the user will depend on this feature is small. Why not just return a zero row value? I may be wrong but I can't imagine a use case that needs to recycle the zero-length vector. Again, only the scalar value is supposed to be recycled.

If there's no clear objection, I'm going to file a PR.

[jangorecki]

@shrektan I like your initial report here, it make sense. Looking forward for PR. As for your recent comment, isn't it a different piece of code to be altered to make this join works? If so then two separate PR could be better.

[shrektan]

I think it's the cause is the same.

If J(character(), 1) returns a zero-row data.table, dt[J(character(), 1), VALUE] should return a zero-length result.

I haven't looked into J's implementation yet. If it's quite different, I will separate the PR. Thanks for reminding this.

[MichaelChirico]

Note that J is commented out on its help page:

# DT[J(...)]                          # J() only for use inside DT[...]

actually, J is another example of NSE-only function in data.table. It's simply converted to list internally:

grep "J" R/data.table.R -n
103:  # J alias for list as well in i, just if the first symbol

[shrektan]

Just to update another example that I was bite in the real code.

library(data.table)
tbl <- data.table(A = 1:3, B = letters[1:3])
tbl1 <- tbl[A > 3L, .(ITEM = 'A>3', A, B)]
tbl1
#> Empty data.table (0 rows and 3 cols): ITEM,A,B
tbl2 <- tbl[A > 3][, .(ITEM = 'A>3', A, B)]
#> Warning in as.data.table.list(jval, .named = NULL): Item 2 has 0 rows but
#> longest item has 1; filled with NA
#> Warning in as.data.table.list(jval, .named = NULL): Item 3 has 0 rows but
#> longest item has 1; filled with NA
tbl2
#>    ITEM  A    B
#> 1:  A>3 NA <NA>

^{Created on 2020-05-11 by the reprex package (v0.3.0)}