Select only rows that exist?

[hadley]

When selecting rows with a numeric index, is there a good way to only select rows that exist? For example, I'd like:

library(data.table)

dt <- data.table(x = 1:2)
dt[1:4, ]
#>     x
#> 1:  1
#> 2:  2
#> 3: NA
#> 4: NA

I need this to match the semantics of dplyr::slice() (tidyverse/dtplyr#10).

I also need to do this by-group, so I came up with this:

library(data.table)

dt <- data.table(x = 1:4, y = c(1, 2, 1, 2))
dt[dt[, na.omit(.I[1:3]), by = "y"]$V1]
#>    x y
#> 1: 1 1
#> 2: 3 1
#> 3: 2 2
#> 4: 4 2

The problem with this approach is that it refers to dt twice; this is problematic if dt is actually a compound expression because it will require dt to be evaluated twice. I could assign it to a variable, but I was hoping to find some way to do it in a single sequence of call.

[franknarf1]

I guess your first example has a typo (displaying 4 rows when both commands pertain to two).

Not elegant but it works for any amount of nesting, which may be useful (ie, same approach applies in dt and .SD, both of which are data.tables):

library(data.table)
dt <- data.table(x = 1:2)
rows = 1:4
dt[rows[rows %in% seq_len(.N)]]
# idiomatic approach:
head(dt, 4)

dt <- data.table(x = 1:4, y = c(1, 2, 1, 2))
rows = 1:3
dt[, .SD[rows[rows %in% seq_len(.N)]], by=y]
# idiomatic approach
dt[, head(.SD, 3), by=y]

The idiomatic approach is head, like in base R. (Also, it is/should be optimized by GForce when there's by=)

You cannot use intersect since slice accepts duplicated values, slice(mtcars, c(1,1,1)), I guess.

[hadley]

Ooops, sorry, fixed.

I don't think %in% alone can work because I forgot to mention another wrinkle: rows might have negative indices (i.e. to select which rows to drop).

[jangorecki]

We could support that nicely with nomatch argument

dt[1:4, nomatch=NULL]

which is currently ignored because it is used only during join

[ColeMiller1]

For right now, dt[, head(.SD, n), by = grp] is only GForce optimized for n = 1L.

Here are some timings - dt[dt[, head(.I, 3L), by = grp]$V1] is fastest for now:

dt <- data.table(x = sample(1e5L, 1e7L, TRUE),
                 y = sample(1e3L, 1e7L, TRUE))

bench::mark(
  dt[dt[, na.omit(.I[1:3]), by = "y"]$V1],
  dt[dt[, .I[1:min(.N, 3L)], by = y]$V1],
  dt[dt[, head(.I, 3L), by = y]$V1],
  dt[, head(.SD, 3L), by = y],
  check = F, # idiomatic way places y column first
  min_iterations = 10L
)

# A tibble: 4 x 13
  expression                                min median `itr/sec` mem_alloc
  <bch:expr>                              <bch> <bch:>     <dbl> <bch:byt>
1 dt[dt[, na.omit(.I[1:3]), by = "y"]$V1] 138ms  149ms      5.45    38.3MB
2 dt[dt[, .I[1:min(.N, 3L)], by = y]$V1]  125ms  129ms      7.58    38.3MB
3 dt[dt[, head(.I, 3L), by = y]$V1]       122ms  125ms      7.82    38.3MB
4 dt[, head(.SD, 3L), by = y]             218ms  227ms      4.17    38.4MB

Regarding slices without groupings, this would be an alternative for now - .I isn't available in the i expression to use head(.I, n):

dt <- data.table(x = 1:2)
dt[1:min(.N, 4L)]

While I do not know what is the preferred behavior, the slice() behavior is different than all of the others:

data.frame(x = 1L, y = 2L)[2:3, ]
#>       x  y
#> NA   NA NA
#> NA.1 NA NA
data.table::data.table(x = 1L, y = 2L)[2:3, ]
#>        x     y
#>    <int> <int>
#> 1:    NA    NA
#> 2:    NA    NA
tibble::tibble(x = 1L, y = 2L)[2:3, ]
#> # A tibble: 2 x 2
#>       x     y
#>   <int> <int>
#> 1    NA    NA
#> 2    NA    NA
dplyr::slice(tibble::tibble(x = 1L, y = 2L), 2:3)
#> # A tibble: 0 x 2
#> # ... with 2 variables: x <int>, y <int>