title: How to make new columns with names from lists which are in one of existing columns in pandas, and assign values from lists from another column?
body: <p>I have a problem with big dataframe *about 1kk rows, 180 columns. It starts with 3 columns. First column contains ids. Second and third contains list in each row - they are connected (first row -first element from list from 1st column is connected with first element from list from 2nd column:</p>
  
  <pre><code>ids | fruits | count |
  
  1 | [grape, apple, banana]  | [7.0, 4.0, 3.0]
  
  2 | [mango, banana, strawberry, grape] | [5.0, 8.0, 15.0, 2.0]
  
  3 | [apple, avocado] | [9.0, 1.0]
  4 | NaN | NaN
  5 | [pummelo] | [12.0]
  </code></pre>
  
  <p>I want to use elements of lists from 'fruits' column, as names of new columns which will have values assigned to row and fruits. But no duplicates of columns, like that:</p>
  
  <pre><code>ids | grape | apple | banana | mango | strawberry | avocado | pummelo
  
  1 | 7.0 | 4.0 | 3.0 | NaN | NaN | NaN | NaN
  
  2 | 2.0 | NaN | 8.0 | 5.0 | 15.0 | NaN | NaN
  
  3 | NaN | 9.0 | NaN | NaN | NaN | 1.0 | NaN
  
  4 | NaN | NaN | NaN | NaN | NaN | NaN | NaN
  
  5 | NaN | NaN | NaN | NaN | NaN | NaN | 12.0
  </code></pre>
  
  <p>Count of unique elements in set (non-duplicate sum of all lists) 'fruits'  are 180, and that is why at the end I want to have 180 columns.</p>
  
  <p>The problem is speed. I tried pandas iterrows(), but when it comes to all 1kk rows that becomes neverending story. Below is the code i tried yet.</p>
  
  <pre><code>#making an example dataframe
  
  import numpy as np
  fruit_df = pd. DataFrame(columns=['ids','fruits','count'])
  ids = [1,2,3,4,5]
  fruits = [['grape', 'apple', 'banana'], ['mango', 'banana', 'strawberry', 'grape'], ['apple', 'avocado'], np.nan, ['pummelo']]
  count = [[7.0, 4.0, 3.0],[5.0, 8.0, 15.0, 2.0], [9.0, 1.0], np.nan, [12.0]]
  
  
  #creating fruits columns in dataframe - this one timing is ok , fine for me (about 15 mins)
  
  fruits_columns=[]
  for row in fruit_df['fruits']:
      if type(row)==list:
          fruits_columns.append(row)
      else:
          fruits_columns.append(list())
  
  import itertools
  all_fruits = list(itertools.chain(*fruits_columns))
  
  all_fruits = set(all_fruits)
  
  for fruit in all_fruits:
      fruit_df[fruit]=np.nan
  
  
  #iterating over the data - here is main problem - takes very, very long time.. works well for this tiny dataset but when it comes to 1000000 rows and 180 columns...
  
  def iter_over_rows(data):
      for index, row in data.iterrows():
          if type(row['fruits'])!=float:
              for cat in range(len(row['fruits'])):       
                  data[row['fruits'][cat]][index] = row['count'][cat]
  </code></pre>
  
  <p>I want to speed up this data processing thing. Thought about making dictionaries with all 180 fruits as keys and them counts as value- but at the end order will be damaged. That would be wonderful if you have any idea how to do that faster. Cheers !</p>