[R-Forge #5323] fread: colClasses does not convert to non-builtin types

[arunsrinivasan]

Submitted by: James Sams; Assigned to: Nobody; R-Forge link

If I set a column to numeric, character, etc., the conversion works as expected. However, if I set it to a user created class, it seems like fread gets confused. It recognises that the column is supposed to be converted, but doesn't do the conversion.

The message generated by fread in this situation (sample code attached) is:

# Detected eol as \n only (no \r afterwards), the UNIX and Mac standard.
# Using line 2 to detect sep (the last non blank line in the first 'autostart') ... none found (see ?fread). 
# Deducing this is a single column input. Otherwise, please specify 'sep' manually.
# First row with 1 fields occurs on line 1 (either column names or first row of data)
# All the fields on line 1 are character fields. Treating as the column names.
# Count of eol after first data row: 2
# Subtracted 1 for last eol and any trailing empty lines, leaving 1 data rows
# Type codes: 0 (first 5 rows)
# Column 1 ('date') was detected as type 'integer' but bumped to 'numeric' as requested by colClasses
#    0.000s (  0%) Memory map (rerun may be quicker)
#    0.000s (  0%) sep and header detection
#    0.000s (  0%) Count rows (wc -l)
#    0.000s (  0%) Column type detection (first, middle and last 5 rows)
#    0.000s (  0%) Allocation of 1x1 result (xMB) in RAM
#    0.000s (  0%) Reading data
#    0.000s (  0%) Allocation for type bumps (if any), including gc time if triggered
#    0.000s (  0%) Coercing data already read in type bumps (if any)
#    0.000s (  0%) Changing na.strings to NA
#    0.000s        Total

[dselivanov]

Strange issue IMHO. It is normal that fread supports only built-in simple types (and parsing is in C, so we don't know anything about user supplied classes). If somebody need to convert/coerce types he should do it himself using := operator.

[randomizedthinking]

I encountered the similar issue that fread doesn't recognize other data types. Here is a code nugget:

require(data.table)
setClass('myDate')
setAs("character", 'myDate', function(from) as.Date(from, format="%d/%m/%Y"))

input <- '1,15/08/2008\n2,23/05/2010'
tmp <- fread(input, colClasses=c('numeric','myDate'), header=FALSE)

print(tmp)
class(tmp$V2)

The type of tmp$V2 is character but rather than myDate. Meanwhile, read.csv function can do the conversion correctly.

[arunsrinivasan]

This issue is open only to implement derived inbuilt classes (parsing date, POSIXct etc.). fread will not understand all custom classes.