run_in_master cannot find correct variable when running in future.apply

[dipterix]

run_in_master cannot find correct variables when running in future.apply.

The envir passed to future by future_lapply is its own function environment. Because run_in_master relies on this environment to search for variable, the variables are almost surely wrong unless variables are assigned to globalenv.

This is a flaw in future.apply package. There's nothing wrong because envir was expected to find globals, not the "executing environment". However, if globals are given to future, is envir important anymore?

Waiting for reply - futureverse/future.apply#70

[dipterix]

• implement futurenow_lapply which resolve this issue by setting environment as options and force replace the future-registered environment when running run_in_master
• If Henrik accepts to add future.envir in his own package, then need to depreciate futurenow_lapply or make it just a rename.

[dipterix]

A failed example

p was discovered as shiny::p instead of the progress object

library(future)
library(futurenow)
library(shiny)

## uncomment to enable debugging
# options('future.debug' = TRUE)
# futurenow::debug_futurenow('.debug.log', reset = TRUE, master_only = FALSE)

ui <- fluidPage(
  actionButton("ok", "Run")
)

server <- function(input, output, session) {

  observeEvent(input$ok, {
    p <- Progress$new(session = session, min = 0, max = 10)
    plan(futurenow)

    on.exit({
      p$close()
      plan(sequential)
    }, add = TRUE)

    future.apply::future_lapply(1:10, function(i){
      # Run something async

      # Run in master session
      run_in_master({
        p$inc(amount = 1, message = 'Running item',
                         detail = sprintf('Processing %d', i))
      }, local_vars = 'i')

      # Run something async again
    })

  })


}

shinyApp(ui, server)

[dipterix]

Created a wrapper for future_lapply

futurenow/R/apply.R

Lines 1 to 23 in 3e7678a

	#' @export
	futurenow_lapply <- function(
	X, FUN, ..., future.envir = parent.frame(), future.stdout = TRUE,
	future.conditions = NULL, future.globals = TRUE, future.packages = NULL,
	future.lazy = FALSE, future.seed = FALSE, future.scheduling = 1,
	future.chunk.size = NULL, future.label = "future_lapply-%d"
	){
	options("futurenow.lapply.running" = TRUE)
	options("futurenow.lapply.environment" = future.envir)
	on.exit({
	options("futurenow.lapply.running" = FALSE)
	options("futurenow.lapply.environment" = NULL)
	}, add = TRUE, after = TRUE)
	future.apply::future_lapply(
	X, FUN, ..., future.stdout = future.stdout,
	future.conditions = future.conditions, future.globals = future.globals,
	future.packages = future.packages, future.lazy = future.lazy,
	future.seed = future.seed, future.scheduling = future.scheduling,
	future.chunk.size = future.chunk.size, future.label = future.label)
	}