Why are video proxies faster than raw clips in the sequencer?
I think this is because of the codec the images are stored in. Compare replay speed from a video with replay speed from an image sequence. It doesn't go that fast, but at least faster than video. A video codec makes random access of certain frames that is needed for playback harder: if the compression uses data from precedent or following frames to reduce file size, Blender will need to load and decode a whole segment to read one frame. If some complicated math is used, it will slow down further.

When generating proxies, they not only have a lower resolution, but they are also in a codec that Blender can load at a decent speed (jpeg images or jpeg avi). The smaller file size allows for faster loading and keeping more data in RAM.

I think there are many reasons for proxy files being faster at playback, but to me, it's the random access to individual frames Blender gets that matters the most unless your video is 4k or your hard drive is at the other end of the world.