title: Pandas explode and drop duplicates for multiple columns
body: <p>I am having some issues when trying to perform <code>explode</code> on multiple (4) columns. The first problem is that I run into <code>MemoryError</code> if I try to explode all the columns at once. There are many duplicates after exploding each column individually so I could use <code>drop_duplicates()</code>, however since there are <code>lists</code> in the columns it raises <code>TypeError: unhashable type: 'list'</code>. If I convert the columns to strings with <code>astype(str)</code> then these columns cannot be used with <code>.explode()</code>. So if I try to <code>pd.eval()</code> the column before performing the second <code>.explode()</code> I get <code>UndefinedVariableError: name 'nan' is not defined</code>. This is the example data set:</p>
  
  <pre><code>    id     col_1      col_2   col_3   col_4 
  0    1 ['a','b']        nan   ['c']     nan   
  1    2       nan  ['d','e']     nan     nan
  2    3     ['f']        nan     nan     nan
  3    4       nan      ['g']     nan     nan 
  4    5       nan        nan   ['h']     nan
  5    6       nan        nan   ['i']   ['j'] 
  </code></pre>
  
  <p>This is the current code:</p>
  
  <pre><code>for i in new_table:
      new_table = new_table.explode(i)
      new_table = new_table.astype(str)
      new_table = new_table.drop_duplicates()
      new_table['col_1'] = pd.eval(new_table['col_1'])
      new_table['col_2'] = pd.eval(new_table['col_2'])
      new_table['col_3'] = pd.eval(new_table['col_3'])
      new_table['col_4'] = pd.eval(new_table['col_4'])
  </code></pre>
  
  <p>The <code>pd.eval()</code> raises <code>UndefiniedVariableError: name 'nan' is not defined</code>. If I remove the last 4 lines, then the columns are interpreted as strings and in the second loop, the <code>explode()</code> doesn't do anything because the input is a string, not a list. However I <strong>must(?)</strong> convert the columns to strings to perform the <code>drop_duplicates()</code>. </p>
  
  <p>Code for recreating example dataset:</p>
  
  <pre><code>new_table = pd.DataFrame({'id':[1,2,3,4,5,6],
                            'col_1':[['a','b'],np.nan,['f'],np.nan,np.nan,np.nan],
                            'col_2':[np.nan,['d','e'],np.nan,['g'],np.nan,np.nan],
                            'col_3':[['c'],np.nan,np.nan,np.nan,['h'],['i']],
                            'col_4':[np.nan,np.nan,np.nan,np.nan,np.nan,['j']]})
  </code></pre>
  
  <p>Expected output:</p>
  
  <pre><code>id     col_1      col_2   col_3   col_4 
  1          a        nan       c     nan
  1          b        nan       c     nan
  2        nan          d     nan     nan
  2        nan          e     nan     nan
  3          f        nan     nan     nan
  4        nan          g     nan     nan
  5        nan        nan       h     nan
  6        nan        nan       i       j
  </code></pre>