This paper presents to the best of our knowledge the first end-to-end object tracking approach which directly maps from raw sensor input to object tracks in sensor space without requiring any feature engineering or system identification in the form of plant or sensor models. Specifically, our system accepts a stream of raw sensor data at one end and, in real-time, produces an estimate of the entire environment state at the output including even occluded objects. We achieve this by framing the problem as a deep learning task and exploit sequence models in the form of recurrent neural networks to learn a mapping from sensor measurements to object tracks. In particular, we propose a learning method based on a form of input dropout which allows learning in an unsupervised manner, only based on raw, occluded sensor data without access to ground-truth annotations. We demonstrate our approach using a synthetic dataset designed to mimic the task of tracking objects in 2D laser data -- as commonly encountered in robotics applications -- and show that it learns to track many dynamic objects despite occlusions and the presence of sensor noise.
@cite_4: In this article we present an approach for robust multi-object tracking. Typically the task of object tracking can be divided into two subtasks: object detection and object labeling. The main focus of the work presented here is on an approach for consistently labeling objects across a series of video frames using neural networks. Due to the specialization of object detection algortihms it is necessary to divide detection and labeling to enhance their individual skills. In the evaluation we show that the developed labeling is robust against occlusions and can handle low object detection rates.
@cite_8: In this paper, we treat tracking as a learning problem of estimating the location and the scale of an object given its previous location, scale, as well as current and previous image frames. Given a set of examples, we train convolutional neural networks (CNNs) to perform the above estimation task. Different from other learning methods, the CNNs learn both spatial and temporal features jointly from image pairs of two adjacent frames. We introduce multiple path ways in CNN to better fuse local and global information. A creative shift-variant CNN architecture is designed so as to alleviate the drift problem when the distracting objects are similar to the target in cluttered environment. Furthermore, we employ CNNs to estimate the scale through the accurate localization of some key points. These techniques are object-independent so that the proposed method can be applied to track other types of object. The capability of the tracker of handling complex situations is demonstrated in many testing sequences.
In contrast to using such hand-designed pipelines we provide a novel end-to-end trainable solution allowing the robot to automatically learn an appropriate belief state representation as well as the corresponding predict and update operations for an environment containing multiple objects with different appearance and potentially very complex behaviour. While we are not the first to leverage the expressive power of neural networks in the context of tracking, prior art in this domain primarily concerned only the detection part of the tracking pipeline @cite_8 , @cite_4 .