[Request] C API documentation ?

[eddelbuettel]

I am contemplating creating data.table objects from Rcpp. We had some luck with (exported) bindings from xts some of which we wrapped up in the RcppXts package (GitHub, CRAN) which provides some guidance as to how to do this in compiled code.

Could something similar be started here? Creating data.frame objects in C++ is not easy---we mostly just stick things in a list and call the R function as.data.frame() on it. I guess I could always call setDT() after that. But are there any C level functions here we could export, one by one, or dozen by the dozen? Happy to help...

[arunsrinivasan]

Great! I've made an attempt at implementing data.table() completely in C, #1415. But at the time, I had some issues with getting it to work with do.call() and cbind() IIRC. For now, setDT() would be the way to go.

But this is an incentive for me to give it another go :-).

[mattdowle]

Sounds good. Looking at setDT() at R level it seems quite general purpose. Maybe a new C level function to turn a list of same-length vectors into a data.table is what you'd need Dirk? e.g. no recycling.

[eddelbuettel]

Thanks for picking up the thread, gentlemen.

There are really two distinct issues:
-- ornaments for a SEXP so that when the user receives it in R it is a data.table (easy, call setDT() via Rcpp::Function())
-- adding some callable interface for the C level to get some of the nice data.table magic without having to got to R (hard(er), not sure how much is feasible)

Longer term point two would be really appealing. Not sure if it feasible.

[eddelbuettel]

Any news? Working on something where this would be nice to have...

[mattdowle]

No news from my side. Happy to give information or pointers to whoever wants to do it. I don't know what you mean by "nice data.table magic" or "really appealing". It's too unclear to me as to what is being asked for here, currently. Maybe someone else understands or can create some examples to illustrate.

[eddelbuettel]

Will fill in a handful of lines of R code I'd rather do at the level once back at work tomorrow.

[eddelbuettel]

How do I do

seDT(d)
d[ , `:=`(a = 2*b +c,  d = log(e))]

at the C level? The other part is effectively creating classed R vector at that level; I want to use those in data.table, but creating them is not a data.table issue.

[eddelbuettel]

I think I just got a regression. We uses to be able to create a data.frame in C++ (via Rcpp) and then set its class attribute to data.table and all was good. See eg this Rcpp Gallery post for an example -- scroll down for getDT().

When I do this now, and I also tried executing setDT from the C++ level, I get

Warning message:
In [.data.table(dat, , :=(pt, as.POSIXct(ts, tz = "America/Chicago"))) :
Invalid .internal.selfref detected and fixed by taking a (shallow) copy of the data.table so that := can add this new column by reference. At an earlier point, this data.table has been copied by R (or been created manually using structure() or similar). Avoid key<-, names<- and attr<- which in R currently (and oddly) may copy the whole data.table. Use set* syntax instead to avoid copying: ?set, ?setnames and ?setattr. Also, in R<=v3.0.2, list(DT1,DT2) copied the entire DT1 and DT2 (R's list() used to copy named objects); please upgrade to R>v3.0.2 if that is biting. If this message doesn't help, please report to data.table issue tracker so the root cause can be fixed.

Is there a chance to avoid this? And how about a C API to create a data.table?

(I can file a new issue if you prefer.)

[MichaelChirico]

confirming that you're testing against the latest development version?

…

On Fri, May 25, 2018, 10:20 PM Dirk Eddelbuettel ***@***.***> wrote: I think I just got a regression. We uses to be able to create a data.frame in C++ (via Rcpp) and then set its class attribute to data.table and all was good. See eg this Rcpp Gallery post for an example <http://gallery.rcpp.org/articles/creating-integer64-and-nanotime-vectors/> -- scroll down for getDT(). When I do this now, and I also tried executing setDT from the C++ level, I get Warning message: In [.data.table(dat, , :=(pt, as.POSIXct(ts, tz = "America/Chicago"))) : Invalid .internal.selfref detected and fixed by taking a (shallow) copy of the data.table so that := can add this new column by reference. At an earlier point, this data.table has been copied by R (or been created manually using structure() or similar). Avoid key<-, names<- and attr<- which in R currently (and oddly) may copy the whole data.table. Use set* syntax instead to avoid copying: ?set, ?setnames and ?setattr. Also, in R<=v3.0.2, list(DT1,DT2) copied the entire DT1 and DT2 (R's list() used to copy named objects); please upgrade to R>v3.0.2 if that is biting. If this message doesn't help, please report to data.table issue tracker so the root cause can be fixed. Is there a chance to avoid this? And how about a C API to create a data.table? (I can file a new issue if you prefer.) — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1694 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHQQdUW1YpNniHAIkPwpr78xt29TJ33gks5t2CgtgaJpZM4IZa_g> .

[eddelbuettel]

Scratch this for now. It hit me at work, but there are more moving pieces.

I just tried the Gallery example at home, where it still works. This may be something else, it may be the nanotime object and its timezone. Ball in my court.

That said, this ticket is still needed :)

[MichaelChirico]

@eddelbuettel any update?