Crash on replacing list column with numeric values

[renkun-ken]

Consider the following minimal example:

library(data.table)

dt <- data.table(id = 1:10)
dt2 <- dt[, .(id, close)]
dt2

User would expect a numeric close column in dt but unfortunately there's none but dt2 will capture close function silently and make that column a list of the same close functions.

dt3 <- data.table(id = 1:10, price = rnorm(10))
dt2[dt3, close := price, on = .(id)]

Then replacing the close columns with a numeric column in another data.table will cause crash:

 *** caught segfault ***
address 0x6, cause 'memory not mapped'

Traceback:
 1: `[.data.table`(dt2, dt3, `:=`(close, price), on = .(id))
 2: dt2[dt3, `:=`(close, price), on = .(id)]

Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace

This crash seems to only happen on Linux but not macOS as I tried on both platforms. Not sure if crashes on Windows.

If it does not crash, the outcome might be unexpected for many users:

> dt2                                                                                                          
       id                                                                 close
    <int>                                                                <list>
 1:     1  1.9046580,-2.7921368, 0.6122573,-1.4355704,-0.9884517, 1.0093862,...
 2:     2  1.9046580,-2.7921368, 0.6122573,-1.4355704,-0.9884517, 1.0093862,...
 3:     3  1.9046580,-2.7921368, 0.6122573,-1.4355704,-0.9884517, 1.0093862,...
 4:     4  1.9046580,-2.7921368, 0.6122573,-1.4355704,-0.9884517, 1.0093862,...
 5:     5  1.9046580,-2.7921368, 0.6122573,-1.4355704,-0.9884517, 1.0093862,...
 6:     6  1.9046580,-2.7921368, 0.6122573,-1.4355704,-0.9884517, 1.0093862,...
 7:     7  1.9046580,-2.7921368, 0.6122573,-1.4355704,-0.9884517, 1.0093862,...
 8:     8  1.9046580,-2.7921368, 0.6122573,-1.4355704,-0.9884517, 1.0093862,...
 9:     9  1.9046580,-2.7921368, 0.6122573,-1.4355704,-0.9884517, 1.0093862,...
10:    10  1.9046580,-2.7921368, 0.6122573,-1.4355704,-0.9884517, 1.0093862,...

[tlapak]

Can confirm that it crashes on windows. Related to #4166 and closed by #4350. (You generally can't change column type while assigning in a join.)

library("data.table")

set.seed(1)

dt <- data.table(id = 1:10)
dt2 <- dt[, .(id, close)]
dt2

dt3 <- data.table(id = 1:10, price = rnorm(10))
dt2[dt3, close := price, on = .(id)]

dt2

#       id      close
#    <int>     <list>
# 1:     1 -0.6264538
# 2:     2  0.1836433
# 3:     3 -0.8356286
# 4:     4   1.595281
# 5:     5  0.3295078
# 6:     6 -0.8204684
# 7:     7  0.4874291
# 8:     8  0.7383247
# 9:     9  0.5757814
# 10:   10 -0.3053884