Modern neural network-based algorithms are able to produce highly accurate depth estimates from stereo image pairs, nearly matching the reliability of measurements from more expensive depth sensors. However, this accuracy comes with a higher computational cost since these methods use network architectures designed to compute and process matching scores across all candidate matches at all locations, with floating point computations repeated across a match volume with dimensions corresponding to both space and disparity. This leads to longer running times to process each image pair, making them impractical for real-time use in robots and autonomous vehicles. We propose a new stereo algorithm that employs a significantly more efficient network architecture. Our method builds an initial match cost volume using traditional matching costs that are fast to compute, and trains a network to estimate disparity from this volume. Crucially, our network only employs per-pixel and two-dimensional convolution operations: to summarize the match information at each location as a low-dimensional feature vector, and to spatially process these cost-signature' features to produce a dense disparity map. Experimental results on the KITTI benchmark show that our method delivers competitive accuracy at significantly higher speeds---running at 48 frames per second on a modern GPU.
@cite_18: We propose a new approach to the correspondence problem that makes use of non-parametric local transforms as the basis for correlation. Non-parametric local transforms rely on the relative ordering of local intensity values, and not on the intensity values themselves. Correlation using such transforms can tolerate a significant number of outliers. This can result in improved performance near object boundaries when compared with conventional methods such as normalized correlation. We introduce two non-parametric local transforms: the rank transform, which measures local intensity, and the census transform, which summarizes local image structure. We describe some properties of these transforms, and demonstrate their utility on both synthetic and real data.
Estimating disparity by finding dense correspondences is challenging due to the presence of smooth regions, repeating textures, specular highlights, and half-occlusions. Stereo algorithms proceed by first computing an initial score of match quality between each pixel in the reference pixels and all candidate matches. These candidates are indexed by a finite discrete set of candidate disparity values common to all pixels---typically integer pixel disparities ranging from zero to some maximum value---and correspondingly, the matching scores are organized in a cost volume'' along the spatial and disparity dimensions. Traditionally, stereo algorithms used hand-crafted similarity metrics for matching that take into account local neighborhoods around pixels for robustness, while also being efficient to compute @cite_18 .