Why Gerber Viewers are slow to display, zoom, and pan?
Gerber files aren't 'images'. They're manufacturing data that need to be rendered into an image each time. For complex PCBs, this can be slow. Things like polygons needs to be rendered. Smooth and rounded corners need to be rendered. Its sort of like taking a vector graphics image and then adding more metadata to it. Gerbv has an option for fast rendering which skips over the smoothening of edges stage. That should help with lag. 

Some CAD software let you specify a fill size option for polygons.  In eagle, its based on the line thickness you use to draw the polygon outline. The thinner this line is,  the more complicated the polygon becomes and the longer it'll take the gerber to render.  This happens because polygons are rendered line by line.