Bone constraint to 2 other bones I have 3 bones (in the same armature)
a sword, a sheath and a hand

So I put 2 constraint (childOf) to the bone of the sword. One, attached to the bone of the sheath and the other to the hand. Now, if I move the bone of the sword (in pose mode) to the hand and set the reverse, the influence of the 2nd constraint will also change the position in a maner that the sword can't come back in place in the Sheath.

That sounds really simple but I have no clue how am I suppose to do that :\
You can animate the influence of both constraints simultaneously. And don't move the sword if it has already set it's offset to parents. Just place your hand to the sword's handle, keyframe sword's 'hand' constraint influence at 0.0, and its 'sheath' constraint influence at 1.0, and then at the next frame, flip the influence, and keyframe them again ('hand' becomes 1.0, and 'sheath' becomes 0.0). You can use the gradual influence switch (not the next frame, but a few frames delay).
If you have to rotate the sword during the battle, all you need to return it to the sheath is to restore it's original rotation first (duplicating keyframe in the dopesheet or copypaste pose for that bone at different frames).
I don't know how does your sheath work, and as you say, the sword jumps into the hand. See attached blend, how I understood it.