python set seems to hold two identical objects
Question: I have two sets with custom objects in them. I take the objects from one set
and add them to the other set with set.update.

Afterwards, it appears that one set contains two identical objects: their hash
is identical, they are == to each other and not != to each other. If I cast
the set to a list and back to a set, I only have one object in the new set. I
have no other threads or processes running which may mutate the state of any
object somehow in the middle.

I could post my **hash** and **eq** but they call multiple other sub-objects
**hash** and **eq** and it would be a lot of code to include.

Instead here is the debugging code I am running and its output:

    
    
    print('old hash', map(hash, node.incoming_edges))
    print('new hash', map(hash, new_node.incoming_edges))
    if len(new_node.incoming_edges) > 1:
        node1, node2 = list(new_node.incoming_edges)
        print('eq', node1 == node2)
        print('ne', node1 != node2)
    print('type', type(node.incoming_edges))
    print('type', type(new_node.incoming_edges))
    
    new_node.incoming_edges.update(node.incoming_edges)
    
    print('combined')
    if len(new_node.incoming_edges) > 1:
        node1, node2 = list(new_node.incoming_edges)
        print('eq', node1 == node2)
        print('ne', node1 != node2)
    
    print('combined hash', map(hash, new_node.incoming_edges))
    print('len', len(new_node.incoming_edges))
    
    new_node.incoming_edges = set(list(new_node.incoming_edges))
    
    print('len', len(new_node.incoming_edges))
    

and the relevant output:

    
    
    old hash [5805087492197093178]
    new hash [5805087492197093178]
    type <type 'set'>
    type <type 'set'>
    combined
    eq True
    ne False
    combined hash [5805087492197093178, 5805087492197093178]
    len 2
    len 1
    

I was thinking that since my objects are recursive graphs, the hash might be
changing by adding them to the sets, however I'm printing the hash before and
after the operation to confirm that the hash is not changing.

How can this possibly be happening? I would be happy to introduce more debug
output, I can reproduce easily.

P.S. Here is some info from pdb while I was trying to understand what is
happening:

    
    
     57                     print('type', type(new_node.incoming_edges))
     58     
     59                     import pdb; pdb.set_trace()
     60     
     61                     new_node.incoming_edges.update(node.incoming_edges)
     62  ->                 new_node.outgoing_edges.update(node.outgoing_edges)
     63                     # new_node.incoming_edges = set(list(new_node.incoming_edges))
     64     
     65                     print('combined')
     66                     if len(new_node.incoming_edges) > 1:
     67                         node1, node2 = list(new_node.incoming_edges)
    (Pdb) !len(new_node.incoming_edges)
    2
    (Pdb) !x, y = new_node.incoming_edges
    (Pdb) x
    <Edge user.id getters={<SQLQuery tables:users; selects:users.last_name; where:{} input_mapping:{'id': 'users.id'}, <SQLQuery tables:users; selects:users.first_name; where:{} input_mapping:{'id': 'users.id'}} setter=None out=False>
    (Pdb) y
    <Edge user.id getters={<SQLQuery tables:users; selects:users.last_name; where:{} input_mapping:{'id': 'users.id'}, <SQLQuery tables:users; selects:users.first_name; where:{} input_mapping:{'id': 'users.id'}} setter=None out=False>
    (Pdb) hash(x)
    -8545778292158950550
    (Pdb) hash(y)
    -8545778292158950550
    (Pdb) x == y
    True
    (Pdb) x != y
    False
    (Pdb) len(set(list(new_node.incoming_edges)))
    1
    (Pdb) len(new_node.incoming_edges)
    2
    


Answer: Psychic debugging: You've got `set` members in `node` that were added before
this code begins, then mutated in a way that alters their hashes. `set` caches
the hash value of each object on insertion and does not rehash under normal
conditions; in fact, copying or updating from `set` to `set` can avoid
rehashing as well, because it can copy the cached hash value directly instead
of rehashing.

You "fixed" the problem when you convert to `list` (which effectively strips
the cached hash values), then back to `set` (which must now rehash all the
elements, which causes it to fix the duplication).

Of course, that's not a real fix. The real fix is to _NEVER_ make mutable
items hashable; follow Python's lead and either use immutable types only, or
allow conversion from mutable to immutable variants, where only the immutable
variants define `__hash__`. If you define `__hash__`, you're signing up for
the rules for hashable types, which include logical immutability (and
immutability for all attributes which are involved in the hash calculation
too).

