How can rms voltage change with a change frequency but Peak to peak voltage still stay constant?
Most multimeters aren't designed for the frequencies you are using.  They are generally intended to be used at around line frequency (50hz or 60Hz.)  Some will also work at higher frequencies, some won't.  Around 10Hz you aren't too far from what the meter expects, so it works OK.  At higher frequencies, it won't work so good.

The multimeter will have filters on its input that limit the frequency range of the input signal.  Check the specifications of your meter to see what range it is guaranteed to work in.  You may find that it is only specified for, say, under 200Hz or maybe just at 60Hz or just at 50Hz.

Here is a good multimeter, and it only goes upto 30kHz.



In response to comment:

Your Fluke 115 is only specified for up to 1kHz when measuring AC RMS voltage.  Above that, anything goes.  So, your meter is misleading you.