In this paper we consider critical motion sequences (CMSs) of rolling-shutter (RS) SfM. Employing an RS camera model with linearized pure rotation, we show that the RS distortion can be approximately expressed by two internal parameters of an "imaginary" camera plus one-parameter nonlinear transformation similar to lens distortion. We then reformulate the problem as self-calibration of the imaginary camera, in which its skew and aspect ratio are unknown and varying in the image sequence. In the formulation, we derive a general representation of CMSs. We also show that our method can explain the CMS that was recently reported in the literature, and then present a new remedy to deal with the degeneracy. Our theoretical results agree well with experimental results; it explains degeneracies observed when we employ naive bundle adjustment, and how they are resolved by our method.
@cite_2: We address the problem of Structure from Motion (SfM) with rolling shutter cameras. We first show that many common camera configurations, e.g. cameras with parallel readout directions, become critical and allow for a large class of ambiguities in multi-view reconstruction. We provide mathematical analysis for one, two and some multi-view cases and verify it by synthetic experiments. Next, we demonstrate that bundle adjustment with rolling shutter cameras, which are close to critical configurations, may still produce drastically deformed reconstructions. Finally, we provide practical recipes how to photograph with rolling shutter cameras to avoid scene deformations in SfM. We evaluate the recipes and provide a quantitative analysis of their performance in real experiments. Our results show how to reconstruct correct 3D models with rolling shutter cameras.
Our study differs from theirs in that we derive a general representation of CMSs, although we assume an RS model of rotation-only camera motion. We also show that our method can explain the same CMS as @cite_2 but in a different way, and then present a new method to cope with it. Another difference is that our derivation needs approximation beyond the approximation of linearized rotation. Thus, rigorously speaking, our results state that, at least in the range where our approximation is effective, RS SfM will suffer from the same degeneracy. However, considering the nature of the employed approximations, it will probably have at least instability beyond this range. These agree well with our experimental results that will be shown later.