Python 'shelve' that writes itself to disk after each operation
Question: I'm doing a project in Python that involves a lot of interactive work with a
persistent dictionary. If I weren't doing interactive development, I could use
contextlib.closing and be fairly confident that the shelf would get written to
disk eventually. But as it stands there's no chunk of code that I can easily
wrap within a 'with' statement. I'd prefer not to have to trust myself to call
close() on my shelf at the end of a session.

The amount of data involved is not large, and I would happily sync a shelf to
disk after each operation. I've found myself writing a wrapper for shelve that
does just that but I'm not a strong enough Python programmer to identify the
correct set of dict methods that I'd need to override. And it seems to me that
if what I'm doing is a good idea, it's probably been done before. What is the
paradigmatically correct way to handle this?

I'll add that I like using shelve because it's a simple module, and it comes
with Python. If possible I'd prefer to avoid doing something that requires
(for example) pulling in a complicated library for dealing with databases.

I'm using WinXP with SP3, Python 2.7.5 via Anaconda 1.6.2 (32-bit), and
running inside Spyder. I can tell by looking at the modified times for the
file backing my shelf that the shelf isn't updating until I call sync or
close.


Answer: What you can do is subclass `DbfilenameShelf` and override `__setitem__` and
`__delitem__` to automatically sync after each change. Something like this
would probably work (untested):

    
    
    from shelve import DbfilenameShelf
    
    class AutoSyncShelf(DbfilenameShelf):
        # default to newer pickle protocol and writeback=True
        def __init__(self, filename, protocol=2, writeback=True):
            DbfilenameShelf.__init__(self, filename, protocol=protocol, writeback=writeback)
        def __setitem__(self, key, value):
            DbfilenameShelf.__setitem__(self, key, value)
            self.sync()
        def __delitem__(self, key):
            DbfilenameShelf.__delitem__(self, key)
            self.sync()
    
    my_shelf = AutoSyncShelf("myshelf")
    

I can't vouch for the performance of this, of course.

