title: Remove Duplicate rows from csv [headers + Content]
body: <p>I have a data set which more more than 100mb in size and also many in number of files. These files have more than 20 columns and about more than 1 million rows. </p>
  
  <p>The main problem with data is: </p>
  
  <ol>
  <li>Headers are repeating -- Duplicate header rows </li>
  <li>Duplicate rows in full i.e. data from all the columns in that particular row is duplicate.</li>
  </ol>
  
  <p>Without bothering about the which column or how many columns .. only need to Keep the first occurrence and then remove the rest. </p>
  
  <p>I did find too many examples but what I am looking for is the input and output both need to be same file. The only reason to seek help is, I want the same file to be edited. </p>
  
  <p>sample Input: Here 
  <a href="https://www.dropbox.com/s/sl7y5zm0ppqfjn6/sample_duplicate.csv?dl=0" rel="nofollow noreferrer">https://www.dropbox.com/s/sl7y5zm0ppqfjn6/sample_duplicate.csv?dl=0</a></p>
  
  <p>Appreciate the help thanks in advance.. </p>