Python: Using a multidimensional multiprocessing.manager.list()
Question: This might not be its intended use, but I would like to know how to use a
multidimensional manager.list(). I can create on just fine, something like
this:

    
    
    from multiprocessing import manager
    
    test = manager.list(manager.list())
    

however when ever I try to access the first element of the test list it
returns the element's value and not its proxy object

    
    
    test[0]  # returns [] and not the proxy, since I think python is running __getitem__.
    

Is there anyway for me to get around this and use the manager.list() in this
way?


Answer: The [`multiprocessing`
documentation](https://docs.python.org/2.7/library/multiprocessing.html#multiprocessing.managers.SyncManager.list)
has a note on this:

> **Note**
>
> Modifications to mutable values or items in dict and list proxies will not
> be propagated through the manager, because the proxy has no way of knowing
> when its values or items are modified. To modify such an item, you can re-
> assign the modified object to the container proxy:
>  
>  
>     # create a list proxy and append a mutable object (a dictionary)
>     lproxy = manager.list()
>     lproxy.append({})
>     # now mutate the dictionary
>     d = lproxy[0]
>     d['a'] = 1
>     d['b'] = 2
>     # at this point, the changes to d are not yet synced, but by
>     # reassigning the dictionary, the proxy is notified of the change
>     lproxy[0] = d
>  

So, the only way to use a multidimensional list is to actually reassign any
changes you make to the second dimension of the list back to the top-level
list, so instead of:

    
    
    test[0][0] = 1
    

You do:

    
    
    tmp = test[0]
    tmp[0] = 1
    test[0] = tmp
    

Not the most pleasant way to do things, but you can probably write some helper
functions to make it a bit more tolerable.

**Edit:**

It seems the reason that you get a plain list back when you append a
`ListProxy` to another `ListProxy` is because of how pickling Proxies works.
`BaseProxy.__reduce__` creates a `RebuildProxy` object, which what actually is
used to unpickle the `Proxy`. `RebuildProxy` looks like this:

    
    
    def RebuildProxy(func, token, serializer, kwds):
        '''
        Function used for unpickling proxy objects.
    
        If possible the shared object is returned, or otherwise a proxy for it.
        '''
        server = getattr(process.current_process(), '_manager_server', None)
    
        if server and server.address == token.address:
            return server.id_to_obj[token.id][0]
        else:
            incref = (
                kwds.pop('incref', True) and
                not getattr(process.current_process(), '_inheriting', False)
                )
            return func(token, serializer, incref=incref, **kwds)
    

As the docstring says, if the unpickling is occuring inside a manager server,
the actual shared object is created, rather than the `Proxy` to it. This is
probably a bug, and [there is actually one
filed](http://bugs.python.org/issue6766) against this behavior already.

