python script to remove reversed repeated lines
Question: I got a python code that removes lines if they are similar when reversed. For
example if I have a document that contains:

    
    
    1,2 3,4
    5,6 7,8
    2,1 4,3
    5,6 8,7
    

After executing the script, the output is

    
    
    5,6 7,8
    2,1 4,3
    5,6 8,7
    

Consider first column is 1,2 and second column is 7,8 for a line then, if
another line contains reversed values for each column as 2,1 AND 8,7 this is
considered as reversed.

However, i noticed that the script doesn't keep the order of the lines. The
lines order are important for me. Also, I need to remove the second similar
reversed line, not the first one.The code is

    
    
    import sys
    
    with open(sys.argv[1]) as inf:
    
        keys = set()
    
        for line in inf:
    
            ports, ips = line.split()
    
            port1, port2 = ports.split(",")
    
            ip1, ip2 = ips.split(",")
    
            if ip1 < ip2:
    
                keys.add((ip1, port1, ip2, port2))
    
            else:
    
                keys.add((ip2, port2, ip1, port1))
    
    with open('results', 'w') as outf:
    
        for result in keys:
    
            outf.write("{1},{3}\t{0},{2}\n".format(*result))
    

Any ideas? Any suggestion if we can do it on bash scripting?

Thanks


Answer: You can use `collections.OrderedDict` here:

    
    
    >>> from collections import OrderedDict
    >>> dic = OrderedDict()
    with open('file.txt') as f:
        for line in f:
            key = tuple(tuple(x.split(',')) for x in line.split())
            rev_key = tuple(x[::-1] for x in key)
            if key not in dic and rev_key not in dic:
                dic[key] = line.strip()
    ...             
    >>> for v in dic.itervalues():
        print v
    ...     
    1,2 3,4
    5,6 7,8
    5,6 8,7
    

