title: Comparing lists in two columns row-wise efficiently
body: <p>When having a Pandas DataFrame like this: </p>
  
  <pre class="lang-py prettyprint-override"><code>import pandas as pd
  import numpy as np
  df = pd.DataFrame({'today': [['a', 'b', 'c'], ['a', 'b'], ['b']], 
                     'yesterday': [['a', 'b'], ['a'], ['a']]})
  </code></pre>
  
  <pre class="lang-py prettyprint-override"><code>                 today        yesterday
  0      ['a', 'b', 'c']       ['a', 'b']
  1           ['a', 'b']            ['a']
  2                ['b']            ['a']                          
  ... etc
  </code></pre>
  
  <p>But with about 100 000 entries, I am looking to find the additions and removals of those lists in the two columns on a row-wise basis.</p>
  
  <p>It is comparable to this question: <a href="https://stackoverflow.com/questions/35616058/pandas-how-to-compare-columns-of-lists-row-wise-in-a-dataframe-with-pandas-not">Pandas: How to Compare Columns of Lists Row-wise in a DataFrame with Pandas (not for loop)?</a>  but I am looking at the differences, and <code>Pandas.apply</code> method seems not to be that fast for such many entries.
  This is the code that I am currently using. <code>Pandas.apply</code> with <code>numpy's setdiff1d</code> method:</p>
  
  <pre class="lang-py prettyprint-override"><code>additions = df.apply(lambda row: np.setdiff1d(row.today, row.yesterday), axis=1)
  removals  = df.apply(lambda row: np.setdiff1d(row.yesterday, row.today), axis=1)
  </code></pre>
  
  <p>This works fine, however it takes about a minute for 120 000 entries. So is there a faster way to accomplish this?</p>