What exactly exactly is the difference between rms and average voltage value
The average value is the mathematical average of the signal. If you have a 1V amplitude sine wave, it periodically oscillates between -1V and +1V, so the average of that sine wave is 0V.

But obviously, you aren't delivering an average of 0W. In reality, you don't have negative voltages. A resistive load (and any load for that matter) simply sees a voltage being applied to it, regardless of its direction. So, how do we mathematically get the actual "average" of the power being applied? 

Square the signal (to make it all positive), take the average of the squared values (mean-square value) then "undo" it (square root it). That's your actual "average" voltage being applied, hence the term Root-Mean-Square.

Well, it turns out that, for a sinusoid, the RMS voltage equates to $$V_{RMS}=\sqrt\frac{V_{max}^2}{2}=\frac{V_{max}}{\sqrt2}$$