Most conventional photometric stereo algorithms inversely solve a BRDF-based image formation model. However, the actual imaging process is often far more complex due to the global light transport on the non-convex surfaces. This paper presents a photometric stereo network that directly learns relationships between the photometric stereo input and surface normals of a scene. For handling unordered, arbitrary number of input images, we merge all the input data to the intermediate representation called observation map that has a fixed shape, is able to be fed into a CNN. To improve both training and prediction, we take into account the rotational pseudo-invariance of the observation map that is derived from the isotropic constraint. For training the network, we create a synthetic photometric stereo dataset that is generated by a physics-based renderer, therefore the global light transport is considered. Our experimental results on both synthetic and real datasets show that our method outperforms conventional BRDF-based photometric stereo algorithms especially when scenes are highly non-convex.
@cite_2: A novel technique called photometric stereo is introduced. The idea of photometric stereo is to vary the direction of incident illumination between successive images, while holding the viewing direction constant. It is shown that this provides sufficient information to determine surface orientation at each image point. Since the imaging geometry is not changed, the correspondence between image points is known a priori. The technique is photometric because it uses the radiance values recorded at a single image location, in successive views, rather than the relative positions of displaced features. Photometric stereo is used in computer-based image understanding. It can be applied in two ways. First, it is a general technique for deter-mining surface orientation at each image point. Second, it is a technique for determining object points that have a particular surface orientation. These applications are illustrated using synthesized examples.
Diverse appearances of real world objects can be encoded by a BRDF @math , which relates the observed intensity @math to the associated surface normal @math , the @math -th incoming lighting direction @math , its intensity @math , and the outgoing viewing direction @math via where @math accounts for attached shadows and @math is an additive error to the model. eq:img_form1 is generally called image formation model . Most photometric stereo algorithms assumed the specific shape of @math and recovered the surface normals of a scene by inversely solving eq:img_form1 from a collection of observations under @math different lighting conditions @math . All the effects that are not represented by a BRDF (image noises, cast shadows, inter-reflections and so on) are typically put together in @math . Note that when the BRDF is Lambertian and the additive error is removed, it is simplified to the traditional Lambertian image formation model @cite_2 .