title: How to remove duplicates from a list in python
body: <p>I have a list: <code>list = ['item1', 'item2', 'item3', 'item4']</code></p>
  
  <p>I want to compare the similarity of all items.</p>
  
  <p>If <code>item2</code> and <code>item3</code> is similar, the result become <code>list = ['item1', 'item2', 'item4']</code></p>
  
  <p>Edit:</p>
  
  <p>Sorry for my confusing question.</p>
  
  <p>list items is set of trigrams. I want to remove the similar item in a list.</p>
  
  <pre><code>list = [('very','beauty','place'),('very','good','place'),('another','trigram','item')]
  </code></pre>
  
  <p>with compute jaccard similarity every pairs-item in that list, if jaccard score of pairs-item > 0.4, i call it similar. In this example, item1 and item2 are similar. The last output i want is:</p>
  
  <pre><code>list = [('very','beauty','place'),('another','trigram','item')]
  </code></pre>
  
  <p>This is the method to calculate jaccard score:</p>
  
  <pre><code>def compute_jaccard_index(set_1, set_2):
     n = len(set_1.intersection(set_2))
     return n / float(len(set_1) + len(set_2) - n)
  </code></pre>