Can a scene be efficiently pre-rendered and displayed along a fixed camera line?
Would Diego Nehab et al's "Accelerating Real-Time Shading with Reverse Reprojection Caching" be the sort of thing you are looking for? 

Paper Link and/or Slide presentation from Graphics Hardware 2007

IIRC, the idea is to reuse a previous frame's rendered image results in the current frame rather than running a, potentially expensive, shader for every pixel.

At the conference, I did ask Diego if he'd considered doing 'bidirectional' reuse (i.e. rendering frames out of order somewhat akin to h.264) but I can't recall his answer. I think in your proposed use case it might be of value.