The mysterious case of the random leakage current
I suspect something to do with the strong magnet (I wouldn't have suspected it, except that it must be an interesting answer or you wouldn't have posed the question).  Perhaps having the magnetic field induces a current in a loop somewhere.  Was the leakage higher when there was more area between the wires connecting to the battery?  Or it could be a current in some other loop.