Remove near-duplicate elements in Python list while preserving variables
Question: I have a list that contains near-duplicate elements, with the exception of a
number that identifies the element. I want to remove all duplicates while
preserving the number of the first element containing a duplicate.

For example, I want to replace `l` with `lnew`:

    
    
    l = ['iter1apple','iter2banana','iter3carrot','iter4apple','iter5orange','iter6banana','iter7mango']
    
    lnew = ['iter1apple','iter2banana','iter3carrot','iter5orange','iter7mango']
    

I'm guessing this has something to do with splitting the number from the rest
of the list element, converting the list to set and using `defaultdict` with
the elements from the split, but I can't figure out how.

Any suggestions would be appreciated.


Answer: If I have understood you correctly, you want to discard the items which end in
one element that is already contained in the list. In that case, you can use a
regular expression and a list to track the elements that have been used:

    
    
    import re
    
    l = ['iter1apple', 'iter2banana', 'iter3carrot', 'iter4apple', 'iter5orange', 'iter6banana', 'iter7mango']
    duplicates = []
    lnew = []
    for item in l:
        match = re.match("^iter\d+(\w+)$", item)
        if match and not match.group(1) in duplicates:
            duplicates.append(match.group(1))
            lnew.append(item)
    
    # lnew = ['iter1apple','iter2banana','iter3carrot','iter5orange','iter7mango']
    

