R crashes when assigning a list with a matrix to a subset of a data.table

[kevinvzandvoort]

Assigning a list with a matrix to a subset of a data.table causes R to crash with the error message:

*** caught segfault ***
address 0x4, cause 'memory not mapped'

I am using R 3.6.3 on platform x86_64-pc-linux-gnu. This happened both when using data.table 1.12.8 and 1.13.0.

Reproducible example:
library(data.table)

y <- data.table(i=c(1:10), m=list(matrix(c(0.1:0,4), 2, 2)))
x <- list(matrix(c(1.1:1.4), 2, 2))

Assignment without subsetting works:
y[, m := x]
y[, "m"] <- x

But when subsetting rows and performing the same operation R terminates after attempting to print the object
y[i>6, m := x]
y

# Output of sessionInfo()
R version 3.6.3 (2020-02-29)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 20.04 LTS

Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0

locale:
[1] LC_CTYPE=en_GB.UTF-8
[2] LC_NUMERIC=C
[3] LC_TIME=en_GB.UTF-8
[4] LC_COLLATE=en_GB.UTF-8
[5] LC_MONETARY=en_GB.UTF-8
[6] LC_MESSAGES=en_GB.UTF-8
[7] LC_PAPER=en_GB.UTF-8
[8] LC_NAME=C
[9] LC_ADDRESS=C
[10] LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_GB.UTF-8
[12] LC_IDENTIFICATION=C

attached base packages:
[1] stats graphics grDevices utils datasets
[6] methods base

loaded via a namespace (and not attached):
[1] compiler_3.6.3 tools_3.6.3

[tlapak]

Fixed by #4350. The matrix is unrelated btw. Though this currently requires you to wrap x in an additional list to get the recycling you're probably expecting:

y <- data.table(i=c(1:10), m=list(matrix(c(0.1:0,4), 2, 2)))
x <- list(matrix(1:4, 2, 2))
y[i>6, m := x]
# Warning message:
# In `[.data.table`(y, i > 6, `:=`(m, x)) :
#   2 column matrix RHS of := will be treated as one vector

y
#        i               m
#    <int>          <list>
# 1:     1 0.1,4.0,0.1,4.0
# 2:     2 0.1,4.0,0.1,4.0
# 3:     3 0.1,4.0,0.1,4.0
# 4:     4 0.1,4.0,0.1,4.0
# 5:     5 0.1,4.0,0.1,4.0
# 6:     6 0.1,4.0,0.1,4.0
# 7:     7               1
# 8:     8               2
# 9:     9               3
# 10:   10               4

y[i>6, m := .(x)]
y
#        i               m
#    <int>          <list>
# 1:     1 0.1,4.0,0.1,4.0
# 2:     2 0.1,4.0,0.1,4.0
# 3:     3 0.1,4.0,0.1,4.0
# 4:     4 0.1,4.0,0.1,4.0
# 5:     5 0.1,4.0,0.1,4.0
# 6:     6 0.1,4.0,0.1,4.0
# 7:     7         1,2,3,4
# 8:     8         1,2,3,4
# 9:     9         1,2,3,4
# 10:   10         1,2,3,4