Illegal opcode error after Bioconductor upgrade

[asenabouth]

Hello,
I recently upgraded my Bioconductor packages and have run into issues with using bplapply while testing my package that's under development. After loading the dependancies my package required, I've isolated it down to certain Bioconductor packages.

BiocParallel works fine when it is the only package loaded. Here is the code I used to test this and the session info.

> library(BiocParallel)
> FUN <- function(x) { round(sqrt(x), 4) }
> ncores <- parallel::detectCores() - 1
> register(MulticoreParam(workers = ncores, progressbar = TRUE), default = TRUE)
> 
> bplapply(1:4, FUN)
  |=====================================================================| 100%

[[1]]
[1] 1

[[2]]
[1] 1.4142

[[3]]
[1] 1.7321

[[4]]
[1] 2

> sessionInfo()
R version 3.4.2 (2017-09-28)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Sierra 10.12.6

Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib

locale:
[1] en_AU.UTF-8/en_AU.UTF-8/en_AU.UTF-8/C/en_AU.UTF-8/en_AU.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] BiocParallel_1.10.1

loaded via a namespace (and not attached):
[1] compiler_3.4.2 parallel_3.4.2

It also works fine with BiocGenerics loaded.

> library(BiocGenerics)
Loading required package: parallel

Attaching package: ‘BiocGenerics’

The following objects are masked from ‘package:parallel’:

    clusterApply, clusterApplyLB, clusterCall, clusterEvalQ,
    clusterExport, clusterMap, parApply, parCapply, parLapply,
    parLapplyLB, parRapply, parSapply, parSapplyLB

The following objects are masked from ‘package:stats’:

    IQR, mad, sd, var, xtabs

The following objects are masked from ‘package:base’:

    anyDuplicated, append, as.data.frame, cbind, colMeans, colnames,
    colSums, do.call, duplicated, eval, evalq, Filter, Find, get,
    grep, grepl, intersect, is.unsorted, lapply, lengths, Map, mapply,
    match, mget, order, paste, pmax, pmax.int, pmin, pmin.int,
    Position, rank, rbind, Reduce, rowMeans, rownames, rowSums,
    sapply, setdiff, sort, table, tapply, union, unique, unsplit,
    which, which.max, which.min

> bplapply(1:4, FUN)
  |=====================================================================| 100%

[[1]]
[1] 1

[[2]]
[1] 1.4142

[[3]]
[1] 1.7321

[[4]]
[1] 2

> sessionInfo()
R version 3.4.2 (2017-09-28)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Sierra 10.12.6

Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib

locale:
[1] en_AU.UTF-8/en_AU.UTF-8/en_AU.UTF-8/C/en_AU.UTF-8/en_AU.UTF-8

attached base packages:
[1] parallel  stats     graphics  grDevices utils     datasets  methods  
[8] base     

other attached packages:
[1] BiocGenerics_0.22.1 BiocParallel_1.10.1

loaded via a namespace (and not attached):
[1] compiler_3.4.2

This is what happens when I load the scater package.

> library(scater)
Loading required package: ggplot2

Attaching package: ‘scater’

The following object is masked from ‘package:stats’:

    filter

> sessionInfo()
R version 3.4.2 (2017-09-28)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Sierra 10.12.6

Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib

locale:
[1] en_AU.UTF-8/en_AU.UTF-8/en_AU.UTF-8/C/en_AU.UTF-8/en_AU.UTF-8

attached base packages:
[1] parallel  stats     graphics  grDevices utils     datasets  methods  
[8] base     

other attached packages:
[1] scater_1.4.0        ggplot2_2.2.1       biomaRt_2.32.1     
[4] Biobase_2.36.2      BiocGenerics_0.22.1 BiocParallel_1.10.1

loaded via a namespace (and not attached):
 [1] tximport_1.4.0       beeswarm_0.2.3       locfit_1.5-9.1      
 [4] reshape2_1.4.2       rhdf5_2.20.0         lattice_0.20-35     
 [7] colorspace_1.3-2     htmltools_0.3.6      stats4_3.4.2        
[10] viridisLite_0.2.0    blob_1.1.0           XML_3.98-1.9        
[13] rlang_0.1.2          glue_1.1.1           DBI_0.7             
[16] bit64_0.9-7          bindrcpp_0.2         matrixStats_0.52.2  
[19] plyr_1.8.4           bindr_0.1            zlibbioc_1.22.0     
[22] stringr_1.2.0        munsell_0.4.3        gtable_0.2.0        
[25] memoise_1.1.0        IRanges_2.10.5       httpuv_1.3.5        
[28] vipor_0.4.5          AnnotationDbi_1.38.2 Rcpp_0.12.13        
[31] xtable_1.8-2         edgeR_3.18.1         scales_0.5.0        
[34] limma_3.32.8         S4Vectors_0.14.7     mime_0.5            
[37] bit_1.1-12           gridExtra_2.3        rjson_0.2.15        
[40] digest_0.6.12        stringi_1.1.5        dplyr_0.7.4         
[43] shiny_1.0.5          grid_3.4.2           tools_3.4.2         
[46] bitops_1.0-6         magrittr_1.5         lazyeval_0.2.0      
[49] RCurl_1.95-4.8       tibble_1.3.4         RSQLite_2.0         
[52] pkgconfig_2.0.1      Matrix_1.2-11        data.table_1.10.4-1 
[55] ggbeeswarm_0.6.0     shinydashboard_0.6.1 assertthat_0.2.0    
[58] viridis_0.4.0        R6_2.2.2             compiler_3.4.2      
> bplapply(1:4, FUN)

 *** caught illegal operation ***
address 0x7fffa0e9aae4, cause 'illegal opcode'

Traceback:
 1: mcfork(detached)
 2: parallel::mcparallel({    con <- NULL    suppressWarnings({        while (is.null(con)) {            con <- tryCatch({                socketConnection(host, port, FALSE, TRUE, "a+b",                   timeout = timeout)            }, error = function(e) {            })        }    })    node <- structure(list(con = con), class = "SOCK0node")    bploop(node)}, detached = TRUE)
 3: .bpforkChild(host, port, rank, timeout)
 4: .bpfork(nnodes, bptimeout(x), .hostname(x), .port(x))
 5: .local(x, ...)
 6: bpstart(BPPARAM, length(X))
 7: bpstart(BPPARAM, length(X))
 8: bplapply(X, FUN, ..., BPREDO = BPREDO, BPPARAM = BPPARAM)
 9: bplapply(X, FUN, ..., BPREDO = BPREDO, BPPARAM = BPPARAM)
10: bplapply(1:4, FUN)
11: bplapply(1:4, FUN)

Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace

I get this same issue when I load packages that use scater, such as scran and my own package. I only get this error when I call bplapply; I'm able to use the functions from the package if they do not call bplapply.

Thanks!

[mtmorgan]

Thanks for the careful bug report. I couldn't reproduce the segfault, but on a similar mac this

library(BiocParallel)
library(scater)
bpstart(MulticoreParam(2))

hangs (becomes unresponsive). It involves a call to mclapply(), and replacing the final line with mclapply(1:2, sqrt) also hangs.

By loading the packages in the sessionInfo() one at a time, the following also hangs

library(parallel)
library(data.table)
mclapply(1:2, sqrt)

So at least for me it looks like an interaction between data.table and the forked processes of the parallel package, on macOS. Can you confirm? This is with

> sessionInfo()
R version 3.4.2 (2017-09-28)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: OS X El Capitan 10.11.6

Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] parallel  stats     graphics  grDevices utils     datasets  methods  
[8] base     

other attached packages:
[1] data.table_1.10.4-2

loaded via a namespace (and not attached):
[1] compiler_3.4.2

From CRAN it looks like this version of data.table was published on 2017-10-12; your version of data.table is one less, from 2017-10-10.

[asenabouth]

Thanks for the prompt and thorough response.
Yes, I was able to replicate the error using my version of data.table.

> library(parallel)
> library(data.table)
data.table 1.10.4.2
  The fastest way to learn (by data.table authors): https://www.datacamp.com/courses/data-analysis-the-data-table-way
  Documentation: ?data.table, example(data.table) and browseVignettes("data.table")
  Release notes, videos and slides: http://r-datatable.com
> mclapply(1:2, sqrt)

 *** caught illegal operation ***
address 0x7fffe1953ae4, cause 'illegal opcode'

I updated my data.table package and still had the error.

> sessionInfo()
R version 3.4.2 (2017-09-28)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Sierra 10.12.6

Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib

locale:
[1] en_AU.UTF-8/en_AU.UTF-8/en_AU.UTF-8/C/en_AU.UTF-8/en_AU.UTF-8

attached base packages:
[1] parallel  stats     graphics  grDevices utils     datasets  methods  
[8] base     

other attached packages:
[1] data.table_1.10.4-2

loaded via a namespace (and not attached):
[1] compiler_3.4.2
> mclapply(1:2, sqrt)

 *** caught illegal operation ***
address 0x7fffe1953ae4, cause 'illegal opcode'

Traceback:
 1: mcfork()
 2: FUN(X[[i]], ...)
 3: lapply(seq_len(cores), inner.do)
 4: mclapply(1:2, sqrt)

Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace

I ran the same functions on a CentOS system and did not run into the same issues. So it's just a MacOS issue?

[mtmorgan]

Yes a macOS issue with MulticoreParam. If you're able to contribute to the upstream bug report Rdatatable/data.table#2418 that would be great.

[mtmorgan]

The data.table fix has propagated to the Bioconductor build system. The TIMEOUTs related to this problem should be gone on the build reports available after about 2pm today, October 29.