Cannot figure out wiring of most basic Honeywell thermostat (heater only)
That will be a fun little project.  I wouldn't get overly concerned about the voltage on those wires.  You're safest bet is to use a small relay to isolate that part of the circuit.  Choose a relay with contacts rated for 120V and 1A and you should be good to go.  This will allow you to be very flexible in your choice of coil voltage and how you drive the load with the Arduino (there are plenty of examples about how to do that).

If you really do want to measure voltage, make sure the thermostat is not requesting heat (set it all the way cold), and set your meter for AC first... if you do get a reading, you can test again with DC to see whether you have an AC or DC source.