What does the moving iron voltmeter measure in a circuit, average voltage or RMS voltage?
A moving iron meter measures true RMS theoretically.

A moving coil meter measures average, or if connected to a DC source it will measure positive and negative depending on the polarity of the source. This means that on an AC signal the positive movements of the needle are countered by the negative movements of the needle and the needle stays where it is (at zero) because it isn't quick enough (too much mass) to follow the signal as it cycles through its waveform.

A moving iron meter doesn't have a permanent magnet - it relies on the magnetic attraction from the current in the coil to "pull" a piece of iron. Positive current in the coil has the same "pull" as negative current.

The two meters are fundamentally different.

But does a moving iron meter measure RMS. Consider the formula for the force exerted on a magnetizable object from current in a loop: -

Force = \$(N\cdot I)^2 \dfrac{μ_0\cdot A}{(2 g^2)}\$

Where: 


μ0 = 4π×10-7
F is the force in Newtons
N is the number of turns of the coil
I is the current in Amps in the coil
A is the area in length units squared (cross sectional area of the coil)
g is the length of the gap between the coil and the iron.


Force is proportional to amps squared and this of course produces a non-linear pull on the iron with respect to linear current but the fact that it is a squared term tells us that the force is proportional to RMS.



This picture was taken from here. As the current through the coil increases, the plunger is drawn further into the coil and the pointer deflects to the right.

Wattmeter - a little more to consider

The moving iron meter is the basis for the dual-moving-coil wattmeter and this measures true power: -



In the wattmeter, what was formerly a moving iron in the volt or ammeter becomes a coil excited with the line AC voltage and because current and voltage have the same frequency (but variable phase due to the load), the two magnetic fields interact to give true power. This sort of device can also be made from Hall-effect sensors and indeed, Landis and Gyr (formerly made electricity meters) patented this "no moving parts" method.