Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

using future_apply inside a future #65

Open
comicfans opened this issue Sep 11, 2020 · 0 comments
Open

using future_apply inside a future #65

comicfans opened this issue Sep 11, 2020 · 0 comments

Comments

@comicfans
Copy link

since fork is unstable in RStudio, I came to another solution, first using multi-session to create a future from main R session, and use multicore to fork in the future expr body(I've modified future to make supportsMulticore return TRUE from none-interactive R), so future_lapply still use fork

plan(multisession)
res = future({
   plan(multicore)
   options(mc.cores = 4)
   options(future.fork.enable=T)
   future_lapply(1:100, function(i){
        return(length(BIG_GLOBAL_VAR))
   })
})

lapply.res = value(res)

I hope such setup should makes big global variable access faster ( should be copied only once) and worker cheaper. with some simple test seems it do complete faster than directly call future_lapply with multisession

my question is : does such setup have some implicit problem ? does future/future.apply expected to work correctly in such setup ? I tried to use such setup with progressr, found that the progressor object can not work, for example

plan(multisession)
with_progress({
  p = progressor(100)
  future({ 
   plan(multicore)
   options(mc.cores = 4)
   options(future.fork.enable=T)
     future_lapply(1:100, function(i){
        p("")
    })
  })
})

is this progressr limitation, or future/future.apply should not be used in such way ? Thanks for advise

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant