How to remove duplicates in a csv file based on two columns?
<p>I have a csv file like this :</p>

<pre><code>column1    column2

john       kerry
adam       stephenson
ashley     hudson
john       kerry
etc..
</code></pre>

<p>I want to  remove duplicates from this file, to get only :</p>

<pre><code>column1    column2

john       kerry
adam       stephenson
ashley     hudson
</code></pre>

<p>I wrote this script that removes duplicates based on lastnames, but I need to remove duplicates based on lastnames AND firstname.   </p>

<pre><code>import csv

reader=csv.reader(open('myfilewithduplicates.csv', 'r'), delimiter=',')
writer=csv.writer(open('myfilewithoutduplicates.csv', 'w'), delimiter=',')

lastnames = set()
for row in reader:
    if row[1] not in lastnames:
        writer.writerow(row)
        lastnames.add( row[1] )
</code></pre>
