Pickling a list consisting of 'n' 2-tuple values
Question: I want to pickle a list as it takes a long time for me to create the list. The
list consists of "n" 2-tuple values where the first tuple value is a large
dictionary(1k to 10k keys) and the second value is a string. N can be as large
as 100s to 1000s. I would like to save such a data structure so that I may
load it on convenience. If the list is human-readable it would be great, but
not at the cost of performance.

I have knowledge about [Shelve](http://docs.python.org/library/shelve.html),
[PyYaml](http://pyyaml.org/),
[cPickle](http://docs.python.org/release/2.5/lib/module-cPickle.html) and
[JSON](https://github.com/jsonpickle/jsonpickle). I am just unsure as to what
to use given my data. Initial reading on various other threads on this website
and internet says that cPickle may be the best choice. Any thoughts from the
gurus here?


Answer: I would use `cPickle`, this works fine,

    
    
    # Dummy data
    from random import randint as r
    
    a,b = 97,123
    
    d = [(dict([(chr(r(a,b)),j) for j in range(1000)]),
           ''.join([chr(r(a,b)) for i in range(5)])) 
                                for j in range(100)]
    
    # Pickle it
    import cPickle as pickle
    
    f = open('store.dat','w')
    pickle.dump(d,f)
    f.close()
    

I would also consider using something like
[dumbdbm](http://docs.python.org/library/dumbdbm.html).

**Added later**

Following on from the example above, you can do something like this,

    
    
    import dumbdbm as dbm
    
    g = dbm.open('store.db')
    g.update([(str(i),pickle.dumps(j)) for i,j in enumerate(d)])
    g.close()
    

