How to determine if TV is on by looking at yellow RCA monitor signal?
First, I'll answer your question about danger to your TV: As long as you connect only high impedance test leads, you pose very little danger to your TV or anything else you want to measure.  This kind of input can be found on a DMM, an oscilloscope, or an analog input pin on an IC - as long as you do not exceed the maximum voltage of the part.  For the Arduino, this means you must be between -0.7 and 5.7V, to avoid turning on the diodes internal to your microcontroller.  As soon as you add discrete components between two digital transceivers, though, you risk a lot.
Also, make sure that your grounds are the same.  0V according to the TV might be much higher than 0V according to your earth-grounded PC.  Test this with a DMM beforehand, measuring between the shield of a USB cable to the shield of the RCA connector.  Put a 100k  resistor between the two shields.  Anything more than a couple volts on this measurement, and you've got trouble.

To answer your first question, though, the output of an NTSC signal in the US is 1V - source this appnote from this page about measuring video signals by Tektronix.   You'll see frequent references to units of IRE, which is a unit for measuring composite video signals that (Unlike voltage) is consistent between various formats and countries.  100 IRE is the difference between white and black.  An NTSC signal will never exceed 140 IRE, which is 1V, so 1 IRE is approximately 7mV for your signal.   

If don't trust this random stranger on the Internet (backed by Tektronix, but how credible are they?),  you can connect your DMM instead of your Arduino, which should read the RMS (Root mean squared) value of what it supposes to be a DC input signal, and should be quite safe up to higher voltages than you want to get near enough to measure (600V?).  The AC amplitude likely won't work, as it's optimized for 60Hz household wiring, but check your manual.