Newly defined S3 generics and methods not passed using plan(multisession)

[zeyuz35]

Describe the bug
A clear and concise description of what the bug is.

Newly defined S3 generics and methods do not work with plan(multisession).

Reproduce example
A reproducible example using R code.

library(doFuture)

registerDoFuture()
plan(multisession)

new_s3_obj <- list(value = 2)
class(new_s3_obj) <- "new_s3"

new_method <- function(x) {
UseMethod("new_method", x)
}

new_method.new_s3 <- function(obj) {
obj$value^2
}

foreach(i = 1:2) %dopar% {
new_method(new_s3_obj)
}

Expected behavior
A clear and concise description of what you expected to happen.

Changing the %dopar% to %do% (sequential), or using plan(multicore) results in the below as expected:

[[1]]
[1] 4

[[2]]
[1] 4

Session information
Please share your session information after the error has occurred so that we also see which packages and versions are involved, e.g.

> sessionInfo()

R version 4.1.0 (2021-05-18)
Platform: aarch64-apple-darwin20 (64-bit)
Running under: macOS Big Sur 11.4

Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/lib/libRlapack.dylib

locale:
[1] en_AU.UTF-8/en_AU.UTF-8/en_AU.UTF-8/C/en_AU.UTF-8/en_AU.UTF-8

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] doFuture_0.12.0 future_1.21.0 foreach_1.5.1

loaded via a namespace (and not attached):
[1] Rcpp_1.0.6 pillar_1.6.1 compiler_4.1.0 iterators_1.0.13 tools_4.1.0
[6] digest_0.6.27 tsibble_1.0.1 lubridate_1.7.10 lifecycle_1.0.0 tibble_3.1.2
[11] gtable_0.3.0 anytime_0.3.9 pkgconfig_2.0.3 rlang_0.4.11 DBI_1.1.1
[16] cli_3.0.0 rstudioapi_0.13 parallel_4.1.0 dplyr_1.0.7 globals_0.14.0
[21] generics_0.1.0 vctrs_0.3.8 grid_4.1.0 tidyselect_1.1.1 glue_1.4.2
[26] listenv_0.8.0 R6_2.5.0 parallelly_1.26.1 fansi_0.5.0 distributional_0.2.2
[31] ggplot2_3.3.5 purrr_0.3.4 farver_2.1.0 tidyr_1.1.3 magrittr_2.0.1
[36] scales_1.1.1 codetools_0.2-18 ellipsis_0.3.2 fabletools_0.3.1 assertthat_0.2.1
[41] colorspace_2.0-2 utf8_1.2.1 munsell_0.5.0 crayon_1.4.1

[HenrikBengtsson]

Thanks for this. Yes, this is "known" and tricky, but not impossible, to automate. I haven't prioritized this one since (a) it works when S3 generics and methods are defined in packages, and (b) it's less common to use them scripts. Having said that, it would be nice to automate this too. This will most likely have to be address by the globals package.

Workaround

Using the approach in https://future.futureverse.org/articles/future-4-issues.html#missing-globals-false-negatives, one can do:

res <- foreach(i = 1:2) %dopar% {
  ## Force needed S3 method(s) to be exported
  new_method.new_s3

  new_method(new_s3_obj)
}

Minimal reproducible example

For future references: Here's a simpler, reproducible example that uses only the future package and that can be used as a package test as well:

library(future)
plan(multisession, workers = 2)

new_method <- function(x) UseMethod("new_method", x)
new_method.new_s3 <- function(x) x^2

obj <- structure(42, class = "new_s3")
f <- future(new_method(obj))
v <- value(f)
v
## Error in UseMethod("new_method", x) : 
##  no applicable method for 'new_method' applied to an object of class "new_s3"

With the above workaround;

f <- future({ new_method.new_s3; new_method(obj) })
v <- value(f)
v
[1] 1764
attr(,"class")
[1] "new_s3"