My Mercury thermostat clicks on when I manipulate it but immediately shuts off
From your description I believe it is a failure of the thermostat or a loose wire.
Is this a standard wet bulb with a bimetallic support?
(A glass globe with mercury and fine wires coming out of 1 end?) .
The most likely cause is the wires to the wet bulb are starting to fail, these thermostats normally last for decades but they do fail. Moving things may make the connection but when you let the pressure off it fails, if it continues to heat while you hold it that would be my first guess, second a loose wire.  I would remove the cover allowing access to the screws and snug them.
Depending on your location you may not be able to get a replacement because of the mercury but they make electronic ones that run on a 9v or a couple of AAA batteries and those are cheaper than the last Hg or mercury one I ordered.
(Some areas will not transport Hg or mercury containing products).