title: why converting list into set in Cython takes so much time?
body: <p><code>huge_list</code> parameter is something like <code>[[12,12,14],[43,356,23]]</code>. And my code to convert list to set is:  </p>
  
  <pre><code>cpdef list_to_set(list huge_list):
      cdef list ids
      cdef list final_ids=[]
      for ids in huge_list:
          final_ids.append(set(ids))
  
      return final_ids
  </code></pre>
  
  <p>I have 2800 list elements, each has 30,000 id. It takes around 19 second. How to improve performance?</p>
  
  <hr>
  
  <p><strong>EDIT 1:</strong><br>
  Instead of <code>set</code> I used <code>unique</code> in <code>numpy</code> as below and <code>numpy</code> speeds up by ~7 seconds:  </p>
  
  <pre><code>df['ids'] = df['ids'].apply(lambda x: numpy.unique(x))
  </code></pre>
  
  <p>Now it takes 14 seconds (Previously it was ~20 seconds). I don't think this time is acceptable yet. :|</p>