Inconsistent j length error even with j returning NULL

[renkun-ken]

The latest dev is broken for use cases where j returns NULL but evaluates list(...) either explicitly or implicitly.

data <- data.table(id = rep(1:2, each = 5), x = rnorm(10), y = rnorm(10))

If j evaluates list(...) with different lengths but finally returns NULL

data[, {
  if (id %% 2 == 0) {
    list(a = 1, b = 2)
  } else {
    list(a = 0)
  }
  NULL
}, keyby = id]

An error still occurs:

Error in `[.data.table`(data, , { :
  j doesn't evaluate to the same number of columns for each group

And if list(...) of the same lengths are evaluated but returns NULL

data[, {
  if (id %% 2 == 0) {
    list(a = 1, b = 2)
  } else {
    list(a = 0, b = 1)
  }
  NULL
}, keyby = id]

A table is incorrectly produced:

   id a b
1:  1 0 1
2:  2 1 2

Even if no explicit list(...) is called, error still occurs, which is a typical use case for group plotting:

col <- c("red", "green")
data[, {
  matplot(x, y, col = col)
  legend("topleft", legend = c("x", "y"), col = col)
  NULL
}, keyby = id]

Error in `[.data.table`(data, , { :
  Supplied 2 items for column 2 of group 1 which has 4 rows. The RHS length must either be 1 (single values are ok) or match the LHS length exactly. If you wish to 'recycle' the RHS please use rep() explicitly to make this intent clear to readers of your code.

I guess this is related to the recently introduced auto-naming j (#3802)?

[MichaelChirico]

yep it must be #3802. thanks for digging into it, I'll have a look

[mattdowle]

Thanks, @renkun-ken! As it happens, I did a full rerun of all revdeps last night since we need to release to CRAN for #4058. All revdeps passed so this is the only known breakage so far.

[MichaelChirico]

I even had the issue written in the comments to the next branch, which also handles the error correctly:

explicit NULL would return NULL, assigning NULL would delete that from the expression

fix en route