How do you exchange an object/mesh between two separate rigged armatures I have a two people walking down a path. One is carrying a bag (guy1) and passes it to the other person (guy2)

I can easily attached the bag to guy1 but I can seem get the exchange right! Im using the Child of constraint to parent the bag to Guy1 hand but it doesn't seem to work further then that.
To hold and drop an object, here is the basic operation to do:


Give the object a Child Of constraint, press Set Inverse so that it stays in place.





Make the parent move from point A (at frame 0) to point B (let's say at frame 20).





The frame before the frame where the dropping is supposed to happen (let say frame 9 as the dropping will happen frame 10), select the child, create a location keyframe and keyframe the Child Of constraint Influence at 1.





The next frame (so, here, frame 10), where the dropping happens, keep the child selected, create a keyframe Visual LocRotScale (or simply Visual Location), so that the new location is the one currently seen on the 3D view. Also keyframe the Child Of constraint Influence at 0 so that the object is not the child anymore.





Now as you give the object to another one, just give this object a second Child Of constraint with the new parent. At frame 9 you'll keyframe the Influence at 0, at frame 10 you'll keyframe the Influence at 1, and now the the child has a new parent.