If first setkey and use by, is it acting the same as we use keyby in data.table

[hope-data-science]

The DT[I,J,BY] is quite perfect for data.table, but keyby is breaking such structure. And my question is as the title, if I never use keyby, and just use setkey/setkeyv and then use by as before, am I getting the exact performance of using keyby?

Thanks.

[jangorecki]

No, you are not getting exact performance. It may be faster or slower depending on your use cases. If you just query once then keyby is likely to be faster, if multiple times then setkey first might be better. In most cases I would expect those differences to be small enough to really care about that, but as always, it is best to measure or your use case, and on your data.

[hope-data-science]

So the situation is, whether I need the group computation multiple times. As stated in the help:

keyby: Same as by, but with an additional setkey() run on the by columns of the result, for convenience. It is common practice to use 'keyby=' routinely when you wish the result to be sorted.

I think it leads me to think keyby is a by and then use setkey. However, I perceive that data.table will do it in sorting order (first keyby) and rearrange back to original order.

Moreover, keyby is not operation by reference, but setkey is (of course). Is that correct?

[jangorecki]

I believe this manual entry simplifies the process to more easily explain its behaviour, by showing how it could be treated from the outside. Internally it is not just setkey at the end of grouping.

Yes, that is correct, keyby will produce new data.table, setkey will update existing object.

[hope-data-science]

Thank you for your professional feedbacks which solves all my puzzles. Looking forward to the next release of data.table~

[jangorecki]

I made it little bit more clear in #4321