[feature request] split out functionality of the fetch function

[ha0ye]

The fetch() function seems to do 2 things:

• download dataset(s) to a temporary directory
• import dataset(s) into a name list

There are existing functions to separately download raw and formatted datasets, but no functions to handle data importing. It seems reasonable to also expose the importing functionality to users and then refactor fetch to be a combined 2-step function that goes through the download and import steps using a temporary directory.

[ethanwhite]

Thanks for the suggestion @ha0ye!

This sounds like a good idea, but I'd like to understand the use case just a little more to help guide development. Is the idea that in some cases you might want to download the data once, but load it repeatedly?

If so do you envision wanting to handle that logic yourself (e.g.,):

if (datafiles %in% list_of_files){
  rdataretriever::load("dataset")
} else {
  rdataretriever::install_csv("dataset")
  rdataretriever::load("dataset")
}

Or would you prefer to have fetch handle this for you, so that fetch looks to see if the csv files have already been created and then loads them if they have and installs and loads if they haven't.

[ha0ye]

Yep, that’s the use case I had in mind.

I think I lean towards expecting a user to handle the logic of installing and loading. You could modify fetch to have additional logic to check downloads, but with the name “fetch”, my default expectation is that it would always be downloading again.

[ethanwhite]

Thanks! One follow up. What do you think about a third alternative: including this behavior in fetch, but as an optional flag:

rdataretriever::fetch("dataset", update_local_data = FALSE)

With the default for update_local_data set to TRUE.

[ha0ye]

Sure, I think that could work!

[ethanwhite]

Thanks!