We propose a novel approach to recovering translucent objects from a single time-of-flight (ToF) depth camera using deep residual networks. When recording translucent objects using the ToF depth camera, their depth values are severely contaminated due to complex light interactions with surrounding environment. While existing methods suggested new capture systems or developed the depth distortion models, their solutions were less practical because of strict assumptions or heavy computational complexity. In this paper, we adopt deep residual networks for modeling the ToF depth distortion caused by translucency. To fully utilize both the local and semantic information of objects, multi-scale patches are used to predict the depth value. Based on the quantitative and qualitative evaluation on our benchmark database, we show the effectiveness and robustness of the proposed algorithm.
@cite_5: This study presents a robust approach to reconstructing a three dimensional (3-D) translucent object using a single time-of-flight depth camera with simple user marks. Because the appearance of translucent objects depends on the light interaction with the surrounding environment, the measurement using depth cameras is considerably biased or invalid. Although several existing methods attempt to model the depth error of translucent objects, their model remains partial because of object assumptions and its sensitivity to noise. In this study, we introduce a ground plane and piece-wise linear surface model as priors and construct a robust 3-D reconstruction framework for translucent objects. These two depth priors are combined with the depth error model built on the time-of-flight principle. Extensive evaluation of various real data reveals that the proposed method substantially improves the accuracy and reliability of 3-D reconstruction for translucent objects.
@cite_21: Translucency introduces great challenges to 3-D acquisition because of complicated light behaviors such as refraction and transmittance. In this paper, we describe the development of a unified 3-D data acquisition framework that reconstructs translucent objects using a single commercial time-of-flight (ToF) camera. In our capture scenario, we record a depth map and intensity image of the scene twice using a static ToF camera; first, we capture the depth map and intensity image of an arbitrary background, and then we position the translucent foreground object and record a second depth map and intensity image with both the foreground and the background. As a result of material characteristics, the translucent object yields systematic distortions in the depth map. We developed a new distance representation that interprets the depth distortion induced as a result of translucency. By analyzing ToF depth sensing principles, we constructed a distance model governed by the level of translucency, foreground depth, and background depth. Using an analysis-by-synthesis approach, we can recover the 3-D geometry of a translucent object from a pair of depth maps and their intensity images. Extensive evaluation and case studies demonstrate that our method is effective for modeling the nonlinear depth distortion due to translucency and for reconstruction of a 3-D translucent object.
Shim and Lee @cite_21 also reported the same analysis, and suggested the depth distortion model based on a ToF depth sensing principle for reconstructing translucent objects. They showed that their model is effective to restore the 3D shape of translucent objects assuming a known background and ignoring the effect of light refraction. Later, Kim and Shim @cite_5 extended the idea of @cite_21 and improved the performance by integrating user interactions. Still, these models assume no refraction, limiting the pose of target object being frontal and the shape being planar. Such assumptions are too strict for practical applications.