Dealing with Random Errors in Sampling Rates?
You might read up on "Aperture Jitter". You are correct that higher frequencies are affected more than lower ones. Here is a good reference:

http://www.analog.com/static/imported-files/tutorials/MT-007.pdf