fread fails when warning is caught: "Previous fread() session was not cleaned up properly. Cleaned up ok at the beginning of this fread() call"

[slazicoicr]

The three lines below work as expected:

fread("will, work\njust, fine\nthank, you", header = FALSE, sep=",", sep2=",")
fread("fails, rather\nbadly, too\nbad", header = FALSE, sep=",", sep2=",")
fread("will, work\njust, fine\nthank, you", header = FALSE, sep=",", sep2=",")

The first and third line read just fine and the second line throws a warning.

The issue happens when the line that throws the warning is wrapped up in a tryCatch block

tryCatch({
  fread("fails, rather\nbadly, too\nbad", header = FALSE, sep=",", sep2=",")
}, warning = function(w) {
  conditionMessage(w)
})

fread("will, work\njust, fine\nthank, you", header = FALSE, sep=",", sep2=",")

Calling the last fread throws a warning message, even though it should work just fine:

Warning message:
In fread("will, work\njust, fine\nthank, you", header = FALSE, sep = ",",  :
  Previous fread() session was not cleaned up properly. Cleaned up ok at the beginning of this fread() call.

sessionInfo:

R version 3.4.4 (2018-03-15)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.4 LTS

Matrix products: default
BLAS: /usr/lib/libblas/libblas.so.3.6.0
LAPACK: /usr/lib/lapack/liblapack.so.3.6.0

locale:
 [1] LC_CTYPE=en_CA.UTF-8       LC_NUMERIC=C               LC_TIME=en_CA.UTF-8       
 [4] LC_COLLATE=en_CA.UTF-8     LC_MONETARY=en_CA.UTF-8    LC_MESSAGES=en_CA.UTF-8   
 [7] LC_PAPER=en_CA.UTF-8       LC_NAME=C                  LC_ADDRESS=C              
[10] LC_TELEPHONE=C             LC_MEASUREMENT=en_CA.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] data.table_1.11.2

loaded via a namespace (and not attached):
[1] compiler_3.4.4 tools_3.4.4  

[mattdowle]

Thanks for reporting.
options(warn=2) was anticipated and does not generate this warning. But trapping warning() via tryCatch() wasn't anticipated unfortunately. Something 'unknown' was anticipated, though, and that's the coping mechanism you're seeing: it cleans itself up upon next call and issues a good warning. It could have been a lot worse.
I don't know of a way to know at R level whether tryCatch(..., warning=) has been specified as something that halts, or not. Any attempt at that is likely to messy and fragile.

One solution might be for DTWARN in fread.c to cache the warning(s) in a private buffer and then call R/Python's warning() on exit after freadCleanup().

[st-pasha]

what if we surround the entire R fread(...) call into one big tryCatch(..., finally=freadCleanup) call?

[mattdowle]

That's a neat idea!
Would be adding tryCatch(..., finally=.Call(CfreadCleanup)) around the .Call(CfreadR) here I guess : https://github.com/Rdatatable/data.table/blob/master/R/fread.R#L101