In this paper, we address the shape-from-shading problem by training deep networks with synthetic images. Unlike conventional approaches that combine deep learning and synthetic imagery, we propose an approach that does not need any external shape dataset to render synthetic images. Our approach consists of two synergistic processes: the evolution of complex shapes from simple primitives, and the training of a deep network for shape-from-shading. The evolution generates better shapes guided by the network training, while the training improves by using the evolved shapes. We show that our approach achieves state-of-the-art performance on a shape-from-shading benchmark.
@cite_7: We relax the long-held and problematic assumption in shape-from-shading (SFS) that albedo must be uniform or known, and address the problem of “shape and albedo from shading” (SAFS). Using models normally reserved for natural image statistics, we impose “naturalness” priors over the albedo and shape of a scene, which allows us to simultaneously recover the most likely albedo and shape that explain a single image. A simplification of our algorithm solves classic SFS, and our SAFS algorithm can solve the intrinsic image decomposition problem, as it solves a superset of that problem. We present results for SAFS, SFS, and intrinsic image decomposition on real lunar imagery from the Apollo missions, on our own pseudo-synthetic lunar dataset, and on a subset of the MIT Intrinsic Images dataset[15]. Our one unified technique appears to outperform the previous best individual algorithms for all three tasks. Our technique allows a coarse observation of shape (from a laser rangefinder or a stereo algorithm, etc) to be incorporated a priori. We demonstrate that even a small amount of low-frequency information dramatically improves performance, and motivate the usage of shading for high-frequency shape (and albedo) recovery.
@cite_34: Since the first shape-from-shading (SFS) technique was developed by Horn in the early 1970s, many different approaches have emerged. In this paper, six well-known SFS algorithms are implemented and compared. The performance of the algorithms was analyzed on synthetic images using mean and standard deviation of depth (Z) error, mean of surface gradient (p, q) error, and CPU timing. Each algorithm works well for certain images, but performs poorly for others. In general, minimization approaches are more robust, while the other approaches are faster.
@cite_40: We examine the shape from shading problem without boundary conditions as a polynomial system. This view allows, in generic cases, a complete solution for ideal polyhedral objects. For the general case we propose a semidefinite programming relaxation procedure, and an exact line search iterative procedure with a new smoothness term that favors folds at edges. We use this numerical technique to inspect shading ambiguities.
@cite_5: 
@cite_16: The objective of this paper is to estimate the orientation of a scene plane from an uncalibrated perspective image under the assumption that the scene is coated with a homogeneous (but unknown) texture. We make the following novel contributions: first, we show that the problem is equivalent to estimating the vanishing line of the plane; second, we show that estimating the two degrees of freedom of this line can be decomposed into two searches each for one parameter; third, we give an algorithm for this estimation which is applicable to both regular and irregular textures. The algorithms do not require that texels are identified explicitly. But once the plane vanishing line has been obtained, then texels locations can be determined, and the geometry of the scene plane computed up to an affine transformation. We give examples of these computations on real images.
@cite_12: We address the problem of recovering shape, albedo, and illumination from a single grayscale image of an object, using shading as our primary cue. Because this problem is fundamentally underconstrained, we construct statistical models of albedo and shape, and define an optimization problem that searches for the most likely explanation of a single image. We present two priors on albedo which encourage local smoothness and global sparsity, and three priors on shape which encourage flatness, outward-facing orientation at the occluding contour, and local smoothness. We present an optimization technique for using these priors to recover shape, albedo, and a spherical harmonic model of illumination. Our model, which we call SAIFS (shape, albedo, and illumination from shading) produces reasonable results on arbitrary grayscale images taken in the real world, and outperforms all previous grayscale “intrinsic image” — style algorithms on the MIT Intrinsic Images dataset.
@cite_11: We develop a framework for extracting a concise representation of the shape information available from diffuse shading in a small image patch. This produces a mid-level scene descriptor, comprised of local shape distributions that are inferred separately at every image patch across multiple scales. The framework is based on a quadratic representation of local shape that, in the absence of noise, has guarantees on recovering accurate local shape and lighting. And when noise is present, the inferred local shape distributions provide useful shape information without over-committing to any particular image explanation. These local shape distributions naturally encode the fact that some smooth diffuse regions are more informative than others, and they enable efficient and robust reconstruction of object-scale shape. Experimental results show that this approach to surface reconstruction compares well against the state-of-art on both synthetic images and captured photographs.
Recovering 3D properties from a single image is one of the most fundamental problems of computer vision. Early works mostly focused on developing analytical solutions and optimization techniques, with zero or minimal learning @cite_16 @cite_34 @cite_5 @cite_12 @cite_7 . Recent successes in this direction include the SIRFS algorithm by Barron and Malik @cite_5 , the local shape from shading method by Xiong al @cite_11 , and polynomial SFS'' algorithm by Ecker and Jepson @cite_40 . All these methods have interpretable, glass box'' models with elegant insights, but in order to maintain analytical tractability, they have to make substantial assumptions that may not hold in unconstrained settings. For example, SIRFS @cite_5 assumes a known object boundary, which is often unavailable in practice. The method by Xiong al assumes quadratically parameterized surfaces, which has difficulties approximating sharp edges or depth discontinuities.