Creating heavily nested python dictionaries in a clean programmatic way
Question: I am using a series of functions to populate a heavily nested dictionary. I am
wondering if there is a cleaner way to do this than just long assignment
string as shown in the example below.

    
    
    outputdict = {}
    outputdict['x']={}
    outputdict['x']['y']={}
    outputdict['x']['y']['total_patients']=len(example_dict.keys())
    outputdict['x']['y']['z']={}
    for variable1 in variable1s:
        outputdict['x']['y']['z'][str(variable1)]={}
        outputdict['x']['y']['z'][str(variable1)]['total_patients']=function_1(example_dict, variable1).count()
        for popn in ['total','male','female']:
            outputdict['x']['y']['z'][str(variable1)][popn]={}
            for age_bucket in np.linspace(40,60,5):
                age_str = str(age_bucket)+'_'+str(age_bucket+5)
                outputdict['x']['y']['z'][str(variable1)][popn][age_str]={}
                outputdict['x']['y']['z'][str(variable1)][popn]["total"]={}
                for res in restypes:
                    if popn == 'total':
                        codelist, ncodes = function_2(function_1(example_dict, variable1), res, age_bucket)
                    else:
                        codelist, ncodes = function_2_gender(function_1(example_dict, variable1), res, age_bucket, popn)
                    outputdict['x']['y']['z'][str(variable1)][popn][age_str][res]={}
                    outputdict['x']['y']['z'][str(variable1)][popn][age_str][res]['total_codes']=ncodes
                    outputdict['x']['y']['z'][str(variable1)][popn][age_str][res]['top_codes']=[]
                    for item in codelist:
                        disp = {"code": item[0][:2], "value":item[0][2], "count":item[1]}
    
                        outputdict['x']['y']['z'][str(variable1)][popn][age_str][res]['top_codes'].append(disp)
    
    
                    codelist, ncodes = list_top_codes(function_1(example_dict, variable1), res)
                    outputdict['x']['y']['z'][str(variable1)][popn]["total"][res]={}
                    outputdict['x']['y']['z'][str(variable1)][popn]["total"][res]['top_codes']=[]
                    for item in codelist:
                        disp = {"code": item[0][:2], "value":item[0][2], "count":item[1]}
                        outputdict['x']['y']['z'][str(variable1)][popn]["total"][res]['top_codes'].append(disp)
    outputdict
    


Answer: You could use
[autovivification](https://en.wikipedia.org/wiki/Autovivification#Python) with
[`defaultdict`](https://docs.python.org/2/library/collections.html#collections.defaultdict).
This would allow you to skip the creation of empty dicts since they would be
automatically created when undefined key is dereferenced:

    
    
    from collections import defaultdict
    
    dd = lambda: defaultdict(dd)
    
    d = dd()
    d['foo']['bar']['foobar'] = 1
    

So your code would look like following:

    
    
    outputdict = dd()
    outputdict['x']['y']['total_patients']=len(example_dict.keys())
    
    for variable1 in variable1s:
        outputdict['x']['y']['z'][str(variable1)]['total_patients']=function_1(example_dict, variable1).count()
    

The other possible improvement would be storing the nested dictionary to
variable so that you wouldn't have to type the full path everywhere:

    
    
    for variable1 in variable1s:
        nested = dd()
        outputdict['x']['y']['z'][str(variable1)]=nested
        nested['total_patients']=function_1(example_dict, variable1).count()
    

