We present a novel 3D face reconstruction technique that leverages sparse photometric stereo (PS) and latest advances on face registration modeling from a single image. We observe that 3D morphable faces approach provides a reasonable geometry proxy for light position calibration. Specifically, we develop a robust optimization technique that can calibrate per-pixel lighting direction and illumination at a very high precision without assuming uniform surface albedos. Next, we apply semantic segmentation on input images and the geometry proxy to refine hairy vs. bare skin regions using tailored filters. Experiments on synthetic and real data show that by using a very small set of images, our technique is able to reconstruct fine geometric details such as wrinkles, eyebrows, whelks, pores, etc, comparable to and sometimes surpassing movie quality productions.
@cite_31: This paper introduces a 3D imaging framework that combines high-resolution photometric stereo and low-resolution depth. Our approach targets imaging scenarios based on either macro-lens photography combined with focal stacking or a large-format camera that are able to image objects with more than 600 samples per mm @math . These imaging techniques allow photometric stereo algorithms to obtain surface normals at resolutions that far surpass corresponding depth values obtained with traditional approaches such as structured-light, passive stereo, or depth-from-focus. Our work offers two contributions for 3D imaging based on these scenarios. The first is a multi-resolution, patched-based surface reconstruction scheme that can robustly handle the significant resolution difference between our surface normals and depth samples. The second is a method to improve the initial normal estimation by using all the available focal information for images obtained using a focal stacking technique.
In a similar vein as ours, @cite_31 combined a low resolution depth with high resolution photometric stereo where the depth map is obtained via structured light. Compared with the structured light results that are highly noisy, the morphable 3D face geometry is smoother but less accurate. We further conduct optimization and semantic segmentations for refinement.