Removing duplicate users from a list using set()
Question: Trying to remove duplicate users from list with set in python. The problem is
that it is not removing the duplicate users:

    
    
    with open ('live.txt') as file: 
            for line in file.readlines(): 
                    word = line.split()
                    users = (word[word.index('user')+1]) 
                            l  = users.split() 
                            l = set(l)
                            l = sorted(l)
                            print " ".join(l)
    

Here's the contents of `live.txt`:

    
    
    Sep 15 04:34:24 li146-252 sshd[13320]: Failed password for invalid user ronda from 212.58.111.170 port 42201 ssh2
    Sep 15 04:34:26 li146-252 sshd[13322]: Failed password for invalid user ronda from 212.58.111.170 port 42330 ssh2
    Sep 15 04:34:28 li146-252 sshd[13324]: Failed password for invalid user ronda from 212.58.111.170 port 42454 ssh2
    Sep 15 04:34:31 li146-252 sshd[13326]: Failed password for invalid user ronda from 212.58.111.170 port 42579 ssh2
    Sep 15 04:34:33 li146-252 sshd[13328]: Failed password for invalid user romero from 212.58.111.170 port 42715 ssh2
    Sep 15 04:34:36 li146-252 sshd[13330]: Failed password for invalid user romero from 212.58.111.170 port 42838 ssh2
    


Answer: You can try a much simpler way as

    
    
    list(set(<Your user list>))
    

This will return list with no duplicate. Python has datatype `set` which is
collection of unique element. So just by typecasting your `list` to `set` will
automatically remove the duplicates

Example:

    
    
    >>> users = ['john', 'mike', 'ross', 'john','obama','mike']
    >>> list(set(users))
    ['mike', 'john', 'obama', 'ross']
    >>>
    

I hope this will solve your problem:

    
    
    import re
    def remove_me():
        all_users = []
        with open ('live.txt') as file:
            for line in file.readlines():
                pattern = re.compile('(.*user\s*)([a-zA-Z0-9]*)')
                stmt = pattern.match(line)
                all_users.append(stmt.groups()[1])
        unique_users = list(set(all_users))
        print unique_users
    
    if __name__ == "__main__":
        remove_me()
    

