j expression on an empty data.table sometimes produce output

[ethanbsmith]

My understanding is that the j expression is evaluated for each row in the data table. Therefore, I expect an empty data.table to produce no output. In the 3 cases below, I don't understand why case 2 produces any output or why it produces warnings. I also don't think that test 2 and 3 producing different output could be the desired/intended result

a <- data.table(foo = 0, bar = 1)
b <- a[1==2] #empty table
print("Test 1")
b[, .(foo, bar)] # expected output
print("Test 2")
b[, .(foo, bar, baz = "Hello")] #Unexpected output
print("Test 3")
b[1==2, .(foo, bar, baz = "Hello")] #expected output

Output:

[1] "Test 1"
Empty data.table (0 rows) of 2 cols: foo,bar
[1] "Test 2"
     baz
1: Hello
Warning messages:
1: In as.data.table.list(jval) :
  Item 1 is of size 0 but maximum size is 1, therefore recycled with 'NA'
2: In as.data.table.list(jval) :
  Item 2 is of size 0 but maximum size is 1, therefore recycled with 'NA'
[1] "Test 3"
Empty data.table (0 rows) of 3 cols: foo,bar,baz

> sessionInfo()
R version 3.4.1 (2017-06-30)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 16299)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252    LC_MONETARY=English_United States.1252 LC_NUMERIC=C                           LC_TIME=English_United States.1252    

attached base packages:
[1] parallel  stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] XML_3.98-1.9         doParallel_1.0.10    iterators_1.0.8      foreach_1.4.4        jsonlite_1.4         curl_2.6             data.table_1.10.5    quantmod_0.4-12      TTR_0.23-2           xts_0.10-0          
[11] zoo_1.8-0            RODBC_1.3-15         plotrix_3.6-6        RevoUtilsMath_10.0.0 RevoUtils_10.0.5     RevoMods_11.0.0      MicrosoftML_1.5.0    mrsdeploy_1.1.2      RevoScaleR_9.2.1     lattice_0.20-35     
[21] rpart_4.1-11         checkpoint_0.4.0    

loaded via a namespace (and not attached):
[1] codetools_0.2-15       CompatibilityAPI_1.1.0 grid_3.4.1             R6_2.2.0               tools_3.4.1            compiler_3.4.1         rtvs_1.0.0.0           mrupdate_1.0.1   

[MichaelChirico]

I think only Test 2 is surprising, and should be simple to fix.

Test 3 is just about "precedence" -- i is evaluated first

[ethanbsmith]

yes, the issue is with Test 2. sorry if that wasn't clear. maybe this is a better example:

a <- data.table(foo = 0, bar = 1)
identical(a[foo == bar  , .(foo, bar, baz = "Hello")], 
          a[foo == bar][, .(foo, bar, baz = "Hello")])

[jan-glx]

In my opinion, data.table should never recycle zero length arguments, data.table(a = numeric(0), b = 1) should return a zero row data.table. Think of the troubles paste is causing with a similar behavior as data.table's current behavior. (Fortunately paste has a recycle0 argument now). See also #4262

[jan-glx]

This has been fixed by now.

library(data.table)
a <- data.table(foo = 0, bar = 1)
b <- a[1==2] #empty table
print("Test 1")
#> [1] "Test 1"
b[, .(foo, bar)] # expected output
#> Empty data.table (0 rows and 2 cols): foo,bar
print("Test 2")
#> [1] "Test 2"
b[, .(foo, bar, baz = "Hello")] # now expected output
#> Empty data.table (0 rows and 3 cols): foo,bar,baz
print("Test 3")
#> [1] "Test 3"
b[1==2, .(foo, bar, baz = "Hello")] #expected output
#> Empty data.table (0 rows and 3 cols): foo,bar,baz

^{Created on 2023-10-20 by the reprex package (v2.0.1)}

Session info

sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.1.0 (2021-05-18)
#>  os       CentOS Linux 7 (Core)
#>  system   x86_64, linux-gnu
#>  ui       X11
#>  language en_US.UTF-8
#>  collate  en_US.UTF-8
#>  ctype    en_US.UTF-8
#>  tz       Europe/Berlin
#>  date     2023-10-20
#>  pandoc   3.1.1 @ /usr/lib/rstudio-server/bin/quarto/bin/tools/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package     * version date (UTC) lib source
#>  cli           3.6.1   2023-03-23 [1] CRAN (R 4.1.0)
#>  data.table  * 1.14.9  2023-07-13 [1] local
#>  digest        0.6.29  2021-12-01 [1] CRAN (R 4.1.0)
#>  evaluate      0.14    2019-05-28 [1] CRAN (R 4.1.0)
#>  fastmap       1.1.0   2021-01-25 [1] CRAN (R 4.1.0)
#>  fs            1.5.2   2021-12-08 [1] CRAN (R 4.1.0)
#>  glue          1.6.2   2022-02-24 [1] CRAN (R 4.1.0)
#>  highr         0.9     2021-04-16 [1] CRAN (R 4.1.0)
#>  htmltools     0.5.5   2023-03-23 [1] CRAN (R 4.1.0)
#>  knitr         1.37    2021-12-16 [1] CRAN (R 4.1.0)
#>  lifecycle     1.0.3   2022-10-07 [1] CRAN (R 4.1.0)
#>  magrittr      2.0.2   2022-01-26 [1] CRAN (R 4.1.0)
#>  purrr         1.0.1   2023-01-10 [1] CRAN (R 4.1.0)
#>  R.cache       0.16.0  2022-07-21 [1] CRAN (R 4.1.0)
#>  R.methodsS3   1.8.1   2020-08-26 [1] CRAN (R 4.1.0)
#>  R.oo          1.24.0  2020-08-26 [1] CRAN (R 4.1.0)
#>  R.utils       2.11.0  2021-09-26 [1] CRAN (R 4.1.0)
#>  reprex        2.0.1   2021-08-05 [1] CRAN (R 4.1.0)
#>  rlang         1.1.1   2023-04-28 [1] CRAN (R 4.1.0)
#>  rmarkdown     2.23    2023-07-01 [1] CRAN (R 4.1.0)
#>  rstudioapi    0.13    2020-11-12 [1] CRAN (R 4.1.0)
#>  sessioninfo   1.2.2   2021-12-06 [1] CRAN (R 4.1.0)
#>  stringi       1.7.6   2021-11-29 [1] CRAN (R 4.1.0)
#>  stringr       1.5.0   2022-12-02 [1] CRAN (R 4.1.0)
#>  styler        1.10.1  2023-06-05 [1] CRAN (R 4.1.0)
#>  vctrs         0.6.2   2023-04-19 [1] CRAN (R 4.1.0)
#>  withr         2.5.0   2022-03-03 [1] CRAN (R 4.1.0)
#>  xfun          0.39    2023-04-20 [1] CRAN (R 4.1.0)
#>  yaml          2.3.4   2022-02-17 [1] CRAN (R 4.1.0)
#> 
#>  [1] /home/gleixner/R/x86_64-pc-linux-gnu-library/4.1.0d
#>  [2] /software/r/4.1.0/lib64/R/library
#> 
#> ──────────────────────────────────────────────────────────────────────────────

[jangorecki]

Thank you for checking that. Let's just add unit test covering that and then issue will be resolved.

[jan-glx]

There already is one.