Solid State Relay Voltage When Off
Had it been really tungsten lamp, the second column would have been 124V, 124V, 0V.

It turned out in the comment section that you had a 13-W compact fluorescent lamp. 

The answer now is simple.

You are seeing a small but non-zero voltage because the impedance of the compact fluorescent lamp is very high (in your case, about 1/4 of the OFF-state resistance of your SSR). You're just creating an AC voltage divider.

In fact, an OFF-state (i.e. when the voltage is very small) compact fluorescent lamp has a very high impedance, because its switching circuitry is OFF, so there is not power delivered to the tube. Therefore you can only measure the leakage of the brige diodes and startup circuitry consumption (if any is present.). Since it's small, the impedance is high.
As soon as the voltage reaches a threshold level, the switching circuitry turns on to power the tube, and the impedance suddenly decreases.

Many low power LED lamps (especially "filed" lamps) have a similar behavior, as they consist of a capacitor dropper, a bridge and some tens LEDs in series. If the applied voltage is smaller than the total drop of all the LEDs, the current consumption will be extremely small and the impedance will be high.