How do I find duplicates within two columns in a csv file, then combine them in Python?
Question: I am working with a large dataset of protein-protein interactions, which I
have in a .csv file. The first two columns are the interacting proteins, and
the order does not matter (ie A/B is the same as B/A, so they are duplicates).
There is also a third column, with a source for where these interactions were
published. Duplicate pairs can be from the same source, or different sources.

For the duplicates that are from two or more sources, how can I combine them,
then in the third column have all of the sources listed for one interaction?
(i.e. for an interaction A/B, the duplicates would be A/B and B/A).

Here is an example of the columns:

    
    
    Interactor A         Interactor B              Source
    A                    B                         Mary (2005)
    C                    D                         John (2004)
    B                    A                         Mary (2005)
    A                    B                         Steve (1993)
    D                    C                         Steve (1993)
    

In this case, i would need

    
    
    Interactor A         Interactor B              Source
    A                    B                         Mary (2005), Steve (1993)
    C                    D                         John (2004), Steve (1993)
    

Thanks!


Answer: You could aggregate them using the **sorted** `tuple` as a dictionary key (to
make `A, B` and `B, A` equivalent, tuples can be used as a dictionary key,
since it's immutable and hashable - lists are not) . Use a
[`set`](https://docs.python.org/2.7/library/stdtypes.html#set) to store the
aggregated values and avoid duplicates.

I'd also throw in
[`defaultdict`](https://docs.python.org/2/library/collections.html#collections.defaultdict)
to make it nicer to aggregate values:

    
    
    from collections import defaultdict
    import csv
    
    # ... read values using a csv reader (assuming name csv_reader)
    
    result = defaultdict(set)
    for row in csv_reader:
        # create same key for `A, B` and `B, A`
        key = tuple(sorted([row[0], row[1]]))
        result[key].add(row[2])
    
    # result should now contain all aggregated values
    

