Title: Rethinking Reprojection: Closing the Loop for Pose-aware ShapeReconstruction from a Single Image,
Abstract: An emerging problem in computer vision is the reconstruction of 3D shape and
pose of an object from a single image. Hitherto, the problem has been addressed
through the application of canonical deep learning methods to regress from the
image directly to the 3D shape and pose labels. These approaches, however, are
problematic from two perspectives. First, they are minimizing the error between
3D shapes and pose labels - with little thought about the nature of this label
error when reprojecting the shape back onto the image. Second, they rely on the
onerous and ill-posed task of hand labeling natural images with respect to 3D
shape and pose. In this paper we define the new task of pose-aware shape
reconstruction from a single image, and we advocate that cheaper 2D annotations
of objects silhouettes in natural images can be utilized. We design
architectures of pose-aware shape reconstruction which re-project the predicted
shape back on to the image using the predicted pose. Our evaluation on several
object categories demonstrates the superiority of our method for predicting
pose-aware 3D shapes from natural images.