LM350 variable voltage regulator SNAFU
Resistors R1 &amp; R2 are constantly loading the output. So, having the output voltage sage due to probing the output with a DMM is suspicious. 

Its likely a bad solder joint somewhere. When you apply pressure with the DMM probes, you break a connection.

It could also be that the output is oscillating and the impedance of you holding the probes plus meter is sufficient to detune whatever is oscillating.