How to detect what element of a nested list has changed? (python)
Question: I have a large 2D (list of lists) list, each element containing a list of
ints, strings and dicts. I would like to be able to determine the 'path' (eg.
[2][3][2]["items"][2] at the worst!) of any element that is modified, and have
this fire upon modification. This list is to big to scan through and see what
has changed! Ideally, I also want a copy of the new element although this can
be found out later.

My first attempt was to create a class, and override its `__setattr__` method:

    
    
    class Notify():
        def __setattr__(self, name, value):
            self.__dict__[name] = value       #Actually assign the value
            print name, value                 #This will tell us when it fires
    

However, the `__setattr__` method only fires when setting a variable that is
not accessed by an index (or key), because this seems to outsource the call to
the contained list()/dict() class not our class.

    
    
    >>> test = Notify()
    >>> test.var = 1          #This is detected
    var 1
    
    >>> test.var = [1,2,3]    #Now let's set it to a list
    var [1, 2, 3]             
    
    >>> test.var[0] = 12      #But when we assign a value via an index, it doesn't fire 
    >>> test.var
    [12, 2, 3]                #However it still assigns the value, so it must be talking to the list itself!
    

So, to summarise, I want (any method really) that tells me at what (list of
indexes/keys) a change has occurred, and this needs to happen as it occurs
because it is too expensive to scan through the whole list. I also can't rely
on the code that modifies the list to provide the details. If this is not
possible to the nth nested list, I could use something that only gave the
first two indexes as the data there would not be too large to scan. Thank you
in advance for your help!

EDIT: Still no joy, though this question [Track changes to lists and
dictionaries in python?](https://stackoverflow.com/questions/8858525/track-
changes-to-lists-and-dictionaries-in-python) seems to be close to what I need.
Unfortunately, I'm not very good with classes and need the help of someone who
is!

EDIT: Had a look at this [Python: Right way to extend
list](https://stackoverflow.com/questions/21196912/python-right-way-to-extend-
list?rq=1) got me thinking that inheriting `list` might be a bad idea. I have
come up with the following code using a proxy class instead. However the
original problem remains, that modifications to nested lists don't log. Would
class composition, not inheritance be a good idea?

    
    
    from UserList import UserList
    
    class NotifyList(UserList):
    
        def __init__(self, initlist=None):
            self.data = []
            if initlist is not None:
                if type(initlist) is list:
                    self.data[:] = initlist
                elif isinstance(initlist, NotifyList):
                    self.data[:] = initlist.data[:]
                else:
                    self.data = list(initlist)
    
        def __setitem__(self, key, item):
            if type(item) is list:
                self.data[key] = NotifyList(item)
            else:
                self.data[key] = item
            print key, item
    
        def append(self, item):
            if type(item) is list:
                self.data.append(NotifyList(item))
            else:
                self.data.append(item)
            print self.index(item), item
    


Answer: You need to create a reporting chain within your (trackable) list of trackable
lists, where each list reports modifications to its parent. In your
`NotifyList` class, add to the constructor an argument for the parent, and an
argument for the ID by which the parent will know the new item -- when the
parent is a list, this will be a list index:

    
    
    class NotifyList(UserList):
        def __init__(self, inilist=None, parent=None, id=None):
            self.parent = parent
            self.id = id
            # remainder of __init__()...
    

When a modifications happens, the parent should be notified. For example in
`__setitem__`:

    
    
    def __setitem__(self, key, item):
        if type(item) is list:
            self.data[key] = NotifyList(item, self, str(key)) # Don't forget the new item's parent
        else:
            self.data[key] = item
        self.alertParent(str(key), str(item)) # Report up the chain instead of printing
    

`alertParent()` is:

    
    
    def alertParent(self, key, item):
        strChange = "[{0}] = {1}".format(key, item)
        self.parent.notifyChange(self.id, strChange)
    

How does `notifyChange()` work?

    
    
    def notifyChange(self, childKey, strChangeInChild):
        strChange = "[{0}]{1}".format(childKey, strChangeInChild)
        self.parent.notifyChange(self.id, strChange)
    

It just propagates the notification up the chain, adding its own ID to the
message.

The only missing link is, what happens at the top of the reporting chain? The
change message should finally be printed. Here is an easy trick to accomplish
this by reusing `alertParent()`:

    
    
    def alertParent(self, key, item):
        if self.parent is None: # I am the root
            print "[{0}]{1}".format(key, item)
        else:
            # remainder of alertParent() shown above...
    ...
    def notifyChange(self, childKey, strChangeInChild):
        if self.parent is None: # I am the root
            self.alertParent(childKey, strChangeInChild) # Actually just prints a change msg
        else:
            # remainder of notifyChange() shown above...
    

I coded this up, the complete version is available
[here](https://drive.google.com/file/d/0B3cAvQF-
GRG3QXRqOFJyb2dJemc/view?usp=sharing) [Google Doc] (there are a couple of
trivial bug fixes with respect to what I have presented above). In action:

    
    
    >>> from test import NotifyList
    >>> top = NotifyList([0]*3, None, None) # Now we have [0 0 0]
    >>> print top
    NList-[0, 0, 0]
    
    >>> top[0] = NotifyList([0]*3, top, 0) # Now we have [ [0 0 0]  0  0 ]
    [0] = NList-[0, 0, 0] #-------------- The tracking msg is fired
    
    >>> print top
    NList-[<test.NotifyList object at 0x0000000002163320>, 0, 0]
    
    >>> top[0][1] = NotifyList([0]*3, top[0], 1) # Now we have [ [[0 0 0] 0 0]  0  0 ]
    [0][1] = NList-[0, 0, 0] #----------- The tracking msg fired again
    
    >>> top[0][1][2] = "this is a string" # Now we have [ [[0 0 "this is a string] 0 0]  0  0 ]
    [0][1][2] = this is a string #------- And another tracking msg
    
    >>> print top[0][1][2]
    this is a string
    
    >>> print top[0][1]
    NList-[0, 0, 'this is a string']
    

