Python and efficient looping of set intersections (using trees)
Question: Below are the distinct paths of attributes and values of a decision tree. If I
were to enumerate the tree of every combination, the tree would be huge.
So...each path of the tree are all of the distinct attributes and values of
leaf node.

If given a list of values to score, i.e find the the node with the most common
elements, I use the below code.

What is the most insanely fast method of trying to accomplish what I want? The
below works but time is of the up-most important so much so that is worthy of
using `c` and importing into python.

Would a tree structure be faster? If so - what structure? would scipy weave be
faster?

    
    
    nodes = {}
    nodes[1] = ['hod=1','hod=2','state=NY','state=LA']
    nodes[2] = ['hod=3','hod=4','state=FL','state=NV']
    nodes[3] = ['hod=5','hod=6','state=WY','state=HI']
    nodes[4] = ['hod=5','hod=6']
    
    
    score = ['hod=6','state=WY','dow=4']
    score_size = len(score)
    
    max_node = -1
    max_len = -1
    for node_id, node in nodes.iteritems():
        this_node_interection_len = len(set(score).intersection(node))
        if this_node_interection_len>max_len:
            max_len = this_node_interection_len
            max_node = node_id   
        #print node_id, len(set(score).intersection(node))
    
    
    print 'max_node',3
    


Answer: It _might_ be faster to store the data in a heap, where
`len(score.intersection(node))` is key value for each node. This way, building
the initial data structure would be a bit slower than making a flat dictionary
would be, but you could quickly retrieve the top several nodes rather than
just the node with the maximum score.

You should also look into using PyPy or something similar to optimize
performance.

