Voltages in loop not summing up to zero
This is expected: RMS values will never add up to zero. It is the instantaneous voltage that adds up to zero. Also clear because RMS values are never negative (by definition!).

To put it slightly different: You need to take into account the relative phases of the voltages. Note, that even then, it will add up to zero only if the signals are ideally sinusoidal. That for example implies that noise needs to be negligible.

Hint: you have to measure with an oscilloscope (with a differential probe!)

EDIT: Ok, here is the behavior when you know the phase. It is the in-phase, and quadrature components that add up to zero. 

The I and Q compoenents are given by

Urms_inphase = Urms * cos(phi)

and 

Urms_quadrature = Urms * sin(phi)

So if you add up Urms_inphase or Urms_quadrature it will equal zero. (Provided you have sinusoidal signals!). It also shows why you have to know the phase phi of each voltage.