title: Attempting to merge 80+ dataframes and crashing R-Studio
body: <p>I'm trying to merge 80+ dataframes in R. These data frames combined have about 4 million rows and 413 columns.</p>
  <p>Is this feasible on a 12 core 16GB machine?</p>
  <p>In the following code, I make an attempt to merge the dataframes with an inner join. When running this code, R-Studio memory usage grows past 16GB, causing R-Studio to crash. Is there a more efficient way to merge? I'm sure this Reduce function is not great.</p>
  <p>Note: dbdataset is a package which acts as a wrapper around this database with approximately 80 tables. To get the dataframe names I used vcdExtra::datasets. This function lists all of the variable names for the dataframes in the package as strings.</p>
  <p>This quickest/easiest way I could think to do this was to send these names to parse test -&gt; eval, shown below. But this crashes R-Studio.</p>
  <pre><code># devtools::install_github(&quot;MohammedFCIS/dbdataset&quot;)
  library(dbdataset)
  library(vcdExtra)
  library(magrittr)
  data &lt;- vcdExtra::datasets(&quot;dbdataset&quot;)
  df &lt;- data$Item %&gt;% 
      lapply( function(i) i %&gt;% parse(text = .) %&gt;% 
      eval()) %&gt;%
      Reduce(function(x, y) merge(x, y), .)
  
  </code></pre>
  <p>Counting columns and rows:</p>
  <pre><code>library(stringr)
  data &lt;- vcdExtra::datasets(&quot;dbdataset&quot;)
  data$dim %&gt;% str_split(&quot;x&quot;) %&gt;% lapply(function(x) x[[2]]) %&gt;% unlist %&gt;% as.numeric %&gt;% sum
  # 413 total columns
  data$dim %&gt;% str_split(&quot;x&quot;) %&gt;% lapply(function(x) x[[1]]) %&gt;% unlist %&gt;% as.numeric %&gt;% sum
  # 4341477 rows
  </code></pre>
  <p>Edit: Thanks to M.Viking for pointing out some of the tables have different ids linking them. For now I updated my code with some suggestions below, but it still crashes. This version is only joining 9 tables though. I also changed to a full-join.</p>
  <pre><code>
  subset &lt;- data$Item  %&gt;% 
      lapply( function(i) i %&gt;% parse(text = .) %&gt;% eval()) %&gt;% 
      lapply(function(i) 'drugbank-id' %in% names(i)) %&gt;% 
      unlist
  
  X &lt;- data$Item[subset] %&gt;% 
      lapply( function(i) i %&gt;% parse(text = .) %&gt;% eval()) %&gt;%
      purrr::reduce( function(x, y) dplyr::full_join(x, y, by = &quot;drugbank-id&quot;))
  </code></pre>