title: Pandas: drop rows based on duplicated values in a list
body: <p>I would like to drop rows within my dataframe based on if a piece of a string is duplicated within that string.  For example, if the string is jkl-ghi-jkl, I would drop this row because jkl is repeated twice.  I figured that creating a list and checking the list for duplicates would be the ideal approach.</p>
  
  <p>My dataframe for this example consist of 1 column and two data points:</p>
  
  <pre><code>    df1 = pd.DataFrame({'Col1' : ['abc-def-ghi-jkl', 'jkl-ghi-jkl-mno'],})
  </code></pre>
  
  <p>My first step I take is to apply a split to my data, and split of "-"</p>
  
  <pre><code>    List = df1['Col1].str.split('-')
      List
  </code></pre>
  
  <p>Which is yields the output:</p>
  
  <pre><code>    0     [abc, def, ghi, jkl]
      1     [jkl, ghi, jkl, mno]
      Name: Col1, dtype: object
  </code></pre>
  
  <p>My second step I take is to convert my output into lists:</p>
  
  <pre><code>    List = List.tolist()
  </code></pre>
  
  <p>Which yields:</p>
  
  <pre><code>    [['abc', 'def', 'ghi', 'jkl'], ['jkl', 'ghi', 'jkl', 'mno']]
  </code></pre>
  
  <p>My last step I wish to accomplish is to compare a full list with a distinct list of unique values:</p>
  
  <pre><code>    len(List) &gt; len(set(List))
  </code></pre>
  
  <p>Which yields the error:</p>
  
  <pre><code>    TypeError: unhashable type: 'list'
  </code></pre>
  
  <p>I am aware that my .tolist() creates a list of 2 series.  Is there a way to convert these series into a list in order to test for duplicates?  I wish to use this piece of code:</p>
  
  <pre><code>    len(List) &gt; len(set(List)
  </code></pre>
  
  <p>with a drop in order to drop all rows with a duplicated value within each cell.  </p>
  
  <p>Is this the correct way of approaching, or is there a simpler way?</p>
  
  <p>My end output should look like:</p>
  
  <pre><code>     Col1
       abc-def-ghi-jkl
  </code></pre>
  
  <p>Because string jkl-ghi-jkl-mno gets dropped due to "jkl" repeating twice</p>