Camera Calibration - why do you have to move the calibration board? I understand the mathematical derivation of Zhang's Method for camera calibration but what I don't understand is why we need to move and rotate the calibration board for several frames in order to get accurate calibration results? One obvious reason is that in order to detect radial distortion you want the control points to be uniformly distributed along the field of view of the camera. However, couldn't we have also just ensured that the calibration board fills the FOV of the camera and just keep it static? Why do we have to move it around?

Thanks!
This is due to the optimization problem being rather high-dimensional (around 11 parameters). With only a single observation of the calibration board, there would be multiple possible combinations of parameters explaining the observed feature point locations (unless a very constrained camera-model is used). Only a sufficient number of sufficiently independent observations will properly constrain the optimization to yield a well-defined answer. Otherwise the routine will converge to some (local) minimum.

Think of this as solving a set of equations with more unknowns than equations. You will be able to find a solution, however, there would be an infinite number of solutions (one or more free parameters).

With respect to a board viewed from the front (frontoparallel orientation), filling the entire image. Now -- is this a calibration board close to the camera viewed with a wide-angle lens (short focal length), or is it a calibration board far away, viewed with a zoom lens (long focal length)? This shows that we need images with the board tilted (foreshortening).

With some constraints, you can calibrate a camera with a single image of two calibration patterns forming e.g. a 90 degree angle, or two perfectly parallel boards at differents depths.

I have written an article on the usual optimization process, which might help you understand how the process could end up not converging to the global solution: https://calib.io/blogs/knowledge-base/camera-calibration

This has little to do with the fixed grid of the imaging sensor as suggested in other answers.