Add elements of two list of dictionaries based on a key value pair match
Question: Given n lists with m dictionaries as their elements, I would like to produce a
new list, with a joined set of dictionaries.

    
    
    l1 = [{"index":'a', "b":2,'c':9}, {"index":'b', "b":3,"c":5}, {"index":'c', "b":8,"c":8}]
    l2 = [{"index":'a', "b":4,'c':8}, {"index":'b', "b":9,"c":10},{"index":None, "b":11,"c":10}]
    

I would like to produce a joined list:

    
    
    l3 = [{"index":'a', "b":6, "c":17}, 
         {"index":'b', "b":12, "c":15}, 
         {"index":'c', "b":8, "c":8},
         {"index":None, "b":11,"c":10}]
    

I have a method that can merge the two lists. But as you can see above, I also
wish to add the elements.

    
    
    def merge_lists(l1, l2, key):
    merged = {}
    for item in l1+l2:
      if item[key] in merged:
        merged[item[key]].update(item)
      else:
        merged[item[key]] = item
    return [val for (_, val) in merged.items()]
    
    l3 = merge_lists(l1,l2,'index')
    

What is the most efficient way to do this in Python?


Answer: You can use a `Counter` for something like this pretty easily...

    
    
    from collections import defaultdict, Counter
    
    def merge_lists(l1, l2):
        d = defaultdict(Counter)
        for sdict in l1 + l2:
            counter = Counter(sdict)
            d[counter.pop('index')] += counter    
    
        lists = []
        for k, v in d.items():
            result = dict(v)
            result['index'] = k
            lists.append(result)
        return lists
    
    l1 = [{"index":'a', "b":2,'c':9}, {"index":'b', "b":3,"c":5}, {"index":'c', "b":8,"c":8}]
    l2 = [{"index":'a', "b":4,'c':8}, {"index":'b', "b":9,"c":10},{"index":None, "b":11,"c":10}]
    print(merge_lists(l1, l2))
    

The great thing about adding `Counter` instances is that it pretty much just
does what you expect. If one counter doesn't have the key, it adds nothing to
the sum, but if both counters have the given key, then their values are added
and used as the resultant value at that key.

* * *

Note that the order of the merged lists is arbitrary (based on the ordering of
the `defaultdict`). If you need to preserve order in some way, you can either
`sort` after the fact or create a default ordered dict which will preserve the
order based on when the `index` was first seen in `l1` or `l2`:

    
    
    class DefaultOrderedDict(collections.OrderedDict):
    
        def __init__(self, default_factory, *args, **kwargs):
            self.default_factory = default_factory
            super(DefaultOrderedDict, self).__init__(*args, **kwargs)
    
        def __missing__(self, key):
            self[key] = self.default_factory()
            return self[key]
    

(There are more "complete" default ordered dicts floating around on
ActiveState and StackOverflow, but this simple one _should_ work for your
problem at hand)

