Merging similar dictionaries in a list together
Question: New to python here. I've been pulling my hair for hours and still can't figure
this out.

I have a list of dictionaries:

    
    
    [ {'FX0XST001.MID5': '195', 'Name': 'Firmicutes', 'Taxonomy ID': '1239', 'Type': 'phylum'}
      {'FX0XST001.MID13': '4929', 'Name': 'Firmicutes', 'Taxonomy ID': '1239','Type': 'phylum'},
      {'FX0XST001.MID6': '826', 'Name': 'Firmicutes', 'Taxonomy ID': '1239', 'Type': 'phylum'},
                                            .
                                            .
                                            .
                                            .
    
      {'FX0XST001.MID6': '125', 'Name': 'Acidobacteria', 'Taxonomy ID': '57723', 'Type': 'phylum'}
      {'FX0XST001.MID25': '70', 'Name': 'Acidobacteria', 'Taxonomy ID': '57723', 'Type': 'phylum'}
      {'FX0XST001.MID40': '40', 'Name': 'Acidobacteria', 'Taxonomy ID': '57723', 'Type': 'phylum'} ]
    

I want to merge the dictionaries in the list based on their Type, Name, and
Taxonomy ID

    
    
      [ {'FX0XST001.MID5': '195', 'FX0XST001.MID13': '4929', 'FX0XST001.MID6': '826', 'Name': 'Firmicutes', 'Taxonomy ID': '1239', 'Type': 'phylum'}
                                            .
                                            .
                                            .
                                            .
    
        {'FX0XST001.MID6': '125', 'FX0XST001.MID25': '70', 'FX0XST001.MID40': '40', 'Name': 'Acidobacteria', 'Taxonomy ID': '57723', 'Type': 'phylum'}]
    

I have the data structure setup like this because I need to write the data to
CSV using csv.DictWriter later.

Would anyone kindly point me to the right direction?


Answer: You can use the groupby function for this:

<http://docs.python.org/library/itertools.html#itertools.groupby>

    
    
    from itertools import groupby
    
    keyfunc = lambda row : (row['Type'], row['Taxonomy ID'], row['Name'])
    
    result = []
    
    data = sorted(data, key=keyfunc)
    for k, g in groupby(data, keyfunc):
        # you can either add the matching rows to the item so you end up with what you wanted
        item = {}        
        for row in g:
            item.update(row)
        result.append(item)
    
        # or you could just add the matched rows as subitems to a parent dictionary
        # which might come in handy if you need to work with just the parts that are
        # different
        item = {'Type': k[0], 'Taxonomy ID' : k[1], 'Name' : k[2], 'matches': [])
        for row in g:
            del row['Type']
            del row['Taxonomy ID']
            del row['Name']
            item['matches'].append(row)
        result.append(item)  
    

