rbind unmatched columns crashes R session (Windows, dev version)

[shrektan]

Test 450 will crash R session with dev version of data.table (but not with CRAN version) on Windows.

R version: 3.6.2
Platform: Windows 10

library(data.table)
DT = data.table(a=1:3,b=4:6)
rbind(DT,list(c=4L,a=7L))

[jangorecki]

Thanks for reporting.
On linux it is

Error in rbindlist(l, use.names, fill, idcol) : 
  Column 1 ['c'] of item 2 is missing in item 1. Use fill=TRUE to fill with NA (NULL for list columns), or use.names=FALSE to ignore column names.

[shrektan]

This line leads to the crash.

data.table/src/rbindlist.c

Lines 210 to 211 in dd7609e

	snprintf(buff, 1000, _("Column %d ['%s'] of item %d is missing in item %d. Use fill=TRUE to fill with NA (NULL for list columns), or use.names=FALSE to ignore column names.%s"),
	w2+1, str, i+1, missi+1, extra );

[shrektan]

I can confirm it's due to the Chinese translation because when I set the language to en, it works well.

[jangorecki]

fy @MichaelChirico

[shrektan]

Looks like we can't use _() with snprintf() on Windows, because when I change the code from

data.table/src/rbindlist.c

Lines 210 to 212 in dd7609e

	snprintf(buff, 1000, _("Column %d ['%s'] of item %d is missing in item %d. Use fill=TRUE to fill with NA (NULL for list columns), or use.names=FALSE to ignore column names.%s"),
	w2+1, str, i+1, missi+1, extra );
	if (usenames==TRUE) error(buff);

to

if (usenames==TRUE) error( _("Column %d ['%s'] of item %d is missing in item %d. Use fill=TRUE to fill with NA (NULL for list columns), or use.names=FALSE to ignore column names.%s"),
              w2+1, str, i+1, missi+1, extra);

The crash will no longer happen.

If that's the case, we may have to check all the combination of snprintf() and _().

[shrektan]

I'm almost sure that snprintf() can't be used with _() on Windows but I don't know the reason.

As you can see that the following code (another place uses snprintf() with _()), on Windows (compared to Mac), although it doesn't crash the R session, it doesn't interpolate the values like %3$d.

library(data.table)
DT = data.table(a=1:3,b=2)
rbind(DT,list(b=5,a=3), use.names = F)

Mac

Windows

[MichaelChirico]

Thanks for diving on this @shrektan. Have asked about this on r-devel, it's a bit above my pay grade.

https://stat.ethz.ch/pipermail/r-devel/2020-April/079397.html

We would drop translation for such messages if there are no suggestions there... I also didn't see any thing googling around for "snprintf" gettext windows...

In the meantime, I tried to replicate your error on Appveyor on the snprintf-windows-test branch, a few different configs of trying to get Appveyor into the Chinese locale didn't work. If you have an idea how to do that could give it a try?

[jangorecki]

What if the size of chinesse characters is so big that it exceeds the size allocated for buff?
Actually that should not be an issue, because buff keeps our english message, unless gettext convert text in buff in place, which is rather not happenning.