title: Fastest way to filter a data.frame list column contents in R / Rcpp
body: <p>I have a data.frame:</p>
  
  <pre><code>df &lt;- structure(list(id = 1:3, vars = list("a", c("a", "b", "c"), c("b", 
  "c"))), .Names = c("id", "vars"), row.names = c(NA, -3L), class = "data.frame")
  </code></pre>
  
  <p>with a list column (each with a character vector):</p>
  
  <pre><code>&gt; str(df)
  'data.frame':   3 obs. of  2 variables:
       $ id  : int  1 2 3
       $ vars:List of 3
        ..$ : chr "a"
        ..$ : chr  "a" "b" "c"
        ..$ : chr  "b" "c"
  </code></pre>
  
  <p>I want to filter the data.frame according to <code>setdiff(vars,remove_this)</code></p>
  
  <pre><code>library(dplyr)
  library(tidyr)
  res &lt;- df %&gt;% mutate(vars = lapply(df$vars, setdiff, "a"))
  </code></pre>
  
  <p>which gets me this:</p>
  
  <pre><code>   &gt; res
        id vars
      1  1     
      2  2 b, c
      3  3 b, c
  </code></pre>
  
  <p>But to get drop the <code>character(0)</code> vars I have to do something like:</p>
  
  <pre><code>res %&gt;% unnest(vars) # and then do the equivalent of nest(vars) again after...
  </code></pre>
  
  <h3>Actual datasets:</h3>
  
  <ul>
  <li>560K rows and 3800K rows that also have 10 more columns (to carry along).</li>
  </ul>
  
  <p>(this is quite slow, which leads to question...)</p>
  
  <h2>What is the Fastest way to do this in <code>R</code>?</h2>
  
  <ul>
  <li>Is there a <code>dplyr</code>/ <code>data.table</code>/ other faster method?</li>
  <li>How to do this with <code>Rcpp</code>?</li>
  </ul>
  
  <h3>UPDATE/EXTENSION:</h3>
  
  <ul>
  <li><p>can the column modification be done in place rather then by copying the <code>lapply(vars,setdiff(...</code> result?</p></li>
  <li><p>what's the most efficient way to filter out for <code>vars == character(0)</code> if it <em>must</em> be a seperate step.</p></li>
  </ul>