Turn off the completed frame presentation
Blender isn't pausing the render to show you the frame it just rendered, 'around a second' is just the time it is taking before it can start rendering the next frame in your scene. As gandalf3 mentioned in the comments, that could be because it's writing the image to disk, doing some pre-processing of the file, reading in textures (or in your case a movieclip) or doing any memory freeing that might have to happen between frames. 

(Note: I don't actually know the specifics of what blender does between frames, I'm just trying to get the point across that blender doesn't delay things on purpose).

If you render blender's default scene of just a cube at a low resolution you will see how quickly the frames advance. There isn't much 'pausing' happening because the scene is so simple that it requires very little processing (or barely any time to write the frame to disk).

However, there is a way you can potentially speed this up a bit, which is by locking the interface while the render is happening. This is done with the lock icon in the 'Render' panel:



This may give a minor speedup as the render result doesn't have to be drawn to the screen while rendering, but bare in mind that it's probably the processing happening that is causing the most delay, not the drawing to the screen.

Maybe if you render from the command line you will get a more detailed log of what is happening at each point in the rendering process.