Video editing workflow - VSE and compositor slows down framerate when previewing?
Don't use overs; there is no cache in the VSE
Don't use heavy codecs. Try AVI-JPEGs or the like.
Use precisely the resolution of the input; don't set the resolution percentage via the render resolution or you will force resampling per frame.