Initialize/Create/Populate a Dict of a Dict of a Dict in Python
Question: I have used dictionaries in python before but I am still new to python. This
time I am using a dictionary of a dictionary of a dictionary... i.e., a three
layer dict, and wanted to check before programming it.

I want to store all the data in this three-layer dict, and was wondering
what'd be an nice pythonic way to initialize, and then read a file and write
to such data structure.

The dictionary I want is of the following type:

    
    
    {'geneid':
    {'transcript_id':
    {col_name1:col_value1, col_name2:col_value2}
    }
    }
    

The data is of this type:

    
    
    geneid\ttx_id\tcolname1\tcolname2\n
    hello\tNR432\t4.5\t6.7
    bye\tNR439\t4.5\t6.7
    

Any ideas on how to do this in a good way?

Thanks!


Answer: First, let's start with the [`csv`](http://docs.python.org/2/library/csv.html)
module to handle parsing the lines:

    
    
    import csv
    with open('mydata.txt', 'rb') as f:
        for row in csv.DictReader(f, delimiter='\t'):
            print row
    

This will print:

    
    
    {'geneid': 'hello', 'tx_id': 'NR432', 'col_name1': '4.5', 'col_name2': 6.7}
    {'geneid': 'bye', 'tx_id': 'NR439', 'col_name1': '4.5', 'col_name2': 6.7}
    

So, now you just need to reorganize that into your preferred structure. This
is almost trivial, except that you have to deal with the fact that the first
time you see a given `geneid` you have to create a new empty `dict` for it,
and likewise for the first time you see a given `tx_id` within a `geneid`. You
can solve that with
[`setdefault`](http://docs.python.org/2/library/stdtypes.html#dict.setdefault):

    
    
    import csv
    genes = {}
    with open('mydata.txt', 'rb') as f:
        for row in csv.DictReader(f, delimiter='\t'):
            gene = genes.setdefault(row['geneid'], {})
            transcript = gene.setdefault(row['tx_id'], {})
            transcript['colname1'] = row['colname1']
            transcript['colname2'] = row['colname2']
    

You can make this a bit more readable with
[`defaultdict`](http://docs.python.org/2/library/collections.html#collections.defaultdict):

    
    
    import csv
    from collections import defaultdict
    from functools import partial
    genes = defaultdict(partial(defaultdict, dict))
    with open('mydata.txt', 'rb') as f:
        for row in csv.DictReader(f, delimiter='\t'):
            genes[row['geneid']][row['tx_id']]['colname1'] = row['colname1']
            genes[row['geneid']][row['tx_id']]['colname2'] = row['colname2']
    

The trick here is that the top-level `dict` is a special one that returns an
empty `dict` whenever it first sees a new key… and that empty `dict` it
returns is itself an empty `dict`. The only hard part is that `defaultdict`
takes a function that returns the right kind of object, and a function that
returns a `defaultdict(dict)` has to be written with a `partial`, `lambda`, or
explicit functions. (There are recipes on ActiveState and modules on PyPI that
will give you an even more general version of this that creates new
dictionaries as needed all the way down, if you want.)

