title: Dropping duplicates within groups only
body: <p>I want to drop duplicates only in particular subsets from a data frame. Under each "spec" in the column "A" I want to drop duplicates, but I want to keep duplicates in the whole data frame (it is possible to have some rows under the first "spec" which are the same as rows under the second "spec", but under "spec" until the next "spec" I want to drop duplicates)</p>
  
  <p>This is the data frame</p>
  
  <p>df</p>
  
  <pre><code>  A          B            C
    spec       first        second
    test       text1        text2
    act        text12       text13
    act        text14       text15
    test       text32       text33
    act        text34       text35
    test       text85       text86
    act        text87       text88
    test       text1        text2
    act        text12       text13
    act        text14       text15
    test       text85       text86
    act        text87       text88
    spec       third        fourth
    test       text1        text2
    act        text12       text13
    act        text14       text15
    test       text85       text86
    act        text87       text88
    test       text1        text2
    act        text12       text13
    act        text14       text15
    test       text85       text86
    act        text87       text88
  </code></pre>
  
  <p>and this is what I want:</p>
  
  <p>df</p>
  
  <pre><code>  A          B            C
    spec       first        second
    test       text1        text2
    act        text12       text13
    act        text14       text15
    test       text32       text33
    act        text34       text35
    test       text85       text86
    act        text87       text88
    spec       third        fourth
    test       text1        text2
    act        text12       text13
    act        text14       text15
    test       text85       text86
    act        text87       text88
  </code></pre>
  
  <p>I could split the data frame in "little" data frames and then in for loop drop duplicates for each of "little" data frames, and finally concatenate them, but I wonder if there is any other solution.</p>
  
  <p>I tried also and succeeded:</p>
  
  <pre><code>dfList = df.index[df["A"] == "spec"].tolist()
  dfList = np.asarray(dfList)
  for dfL in dfList:
        idx = np.where(dfList == dfL)
        if idx[0][0]!=(len(dfList)-1):
              df.loc[dfList[idx[0][0]]:dfList[idx[0][0]+1]-1]
                       = df.loc[dfList[idx[0][0]]:dfList[idx[0][0]+1]-1].drop_duplicates()
        else:
              df.loc[dfList[idx[0][0]]:] = df.loc[dfList[idx[0][0]]:].drop_duplicates()
  </code></pre>
  
  <p>EDIT:
  I have to add this to the end:</p>
  
  <blockquote>
    <p>df.dropna(how='all', inplace=True)</p>
  </blockquote>
  
  <p>But I just wonder if there is any other solution.</p>