fread read error “Expected sep (' ') but..” with partially quoted field

[geotheory]

fread is having difficulty with a tab-separated file that features lines that are partially quoted. To illustrate (str_works confirms embedded quotes as the cause):

str_fails = 'L1\tsome\tunquoted\tstuff\nL2\tsome\t"half" quoted\tstuff\nL3\tthis\t"should work"\tok thought'
str_works = gsub('"', '', str_fails)
fread(str_works, sep='\t', header=F, skip=0L)
#    V1   V2          V3         V4
#1: L1 some    unquoted      stuff
#2: L2 some half quoted      stuff
#3: L3 this should work ok thought
fread(str_fails, sep='\t', header=F, skip=0L)
# Error in fread(str_fails, sep = "\t", header = F, skip = 0L) : 
#   Expected sep (' ') but 
# ' ends field 3 on line 1 when detecting types: L2 some    "half" quoted   stuff`

Would it be possible to add a quote argument to mitigate, or alternatively an automated way to identify embedded quotes during the quote classification code?

[arunsrinivasan]

Thanks for the report. quote = "" is implemented. Either use that, or wrap your file with "" around character columns so that quote = "\"" can then read it properly. See ?fread quote argument.

[geotheory]

I see the example above produces no error in latest version. Thanks for that. Confused by quote = "" is implemented though - I get unused argument (quote = ..) for everything I've tried..? Do you mean it's a valid argument for the function? I can't see any reference in the documentation to it.

[arunsrinivasan]

I can only recommend to uninstall with remove.packages() and reinstall and try again. Have a look at the Readme and commit history if it's confusing.

[geotheory]

Cheers

[arunsrinivasan]

Removed quote argument with a better fix (slightly more robust) for quotes. Please upgrade and test.

[geotheory]

Well it works on my example. Is this package supposed to be able to install from Github? When I try I get following error on library():

Loading required package: data.table Error in get(method, envir = home) : lazy-load database '/Users/user/Documents/R/library/3.1/data.table/R/data.table.rdb' is corrupt In addition: Warning messages: 1: In .registerS3method(fin[i, 1], fin[i, 2], fin[i, 3], fin[i, 4], : restarting interrupted promise evaluation 2: In get(method, envir = home) : restarting interrupted promise evaluation 3: In get(method, envir = home) : internal error -3 in R_decompress1

[jangorecki]

@geotheory the latest version was just published on CRAN yesterday. Now the github version is the same to the CRAN so you can basically install most recent version from CRAN. As the package binaries may not be yet compiled on CRAN you can use install.packages("data.table", type="source"). PS. Inability to install pkg from github should goes as another issue, including steps to reproduce (as there are many ways to install pkgs), session info, etc.

[geotheory]

OK, thanks again.