Global optimization algorithms have shown impressive performance in data-association-based multi-object tracking, but handling online data remains a difficult hurdle to overcome. In this paper, we present a hybrid data association framework with a min-cost multi-commodity network flow for robust online multi-object tracking. We build local target-specific models interleaved with global optimization of the optimal data association over multiple video frames. More specifically, in the min-cost multi-commodity network flow, the target-specific similarities are online learned to enforce the local consistency for reducing the complexity of the global data association. Meanwhile, the global data association taking multiple video frames into account alleviates irrecoverable errors caused by the local data association between adjacent frames. To ensure the efficiency of online tracking, we give an efficient near-optimal solution to the proposed min-cost multi-commodity flow problem, and provide the empirical proof of its sub-optimality. The comprehensive experiments on real data demonstrate the superior tracking performance of our approach in various challenging situations.
@cite_28: Pedestrians follow different trajectories to avoid obstacles and accommodate fellow pedestrians. Any autonomous vehicle navigating such a scene should be able to foresee the future positions of pedestrians and accordingly adjust its path to avoid collisions. This problem of trajectory prediction can be viewed as a sequence generation task, where we are interested in predicting the future trajectory of people based on their past positions. Following the recent success of Recurrent Neural Network (RNN) models for sequence prediction tasks, we propose an LSTM model which can learn general human movement and predict their future trajectories. This is in contrast to traditional approaches which use hand-crafted functions such as Social forces. We demonstrate the performance of our method on several public datasets. Our model outperforms state-of-the-art methods on some of these datasets. We also analyze the trajectories predicted by our model to demonstrate the motion behaviour learned by our model.
@cite_42: The majority of existing solutions to the Multi-Target Tracking (MTT) problem do not combine cues over a long period of time in a coherent fashion. In this paper, we present an online method that encodes long-term temporal dependencies across multiple cues. One key challenge of tracking methods is to accurately track occluded targets or those which share similar appearance properties with surrounding objects. To address this challenge, we present a structure of Recurrent Neural Networks (RNN) that jointly reasons on multiple cues over a temporal window. Our method allows to correct data association errors and recover observations from occluded states. We demonstrate the robustness of our data-driven approach by tracking multiple targets using their appearance, motion, and even interactions. Our method outperforms previous works on multiple publicly available datasets including the challenging MOT benchmark.
@cite_44: 
@cite_43: Multiple Object Tracking (MOT) is an important computer vision task which has gained increasing attention due to its academic and commercial potential. Although different kinds of approaches have been proposed to tackle this problem, it still has many issues unsolved. For example, factors such as continuous appearance changes and severe occlusions result in difficulties for the task. In order to help the readers understand and learn this topic, we contribute a comprehensive and systematic review. We review the recent advances in various aspects about this topic and propose some interesting directions for future research. To our best knowledge, there has not been any review about this topic in the community. The main contribution of this review is threefold: 1) All key aspects in the multiple object tracking system, including what scenarios the researchers are working on, how their work can be categorized, what needs to be considered when developing a MOT system and how to evaluate a MOT system, are discussed in a clear structure. This review work could not only provide researchers, especially new comers to the topic of MOT, a general understanding of the state-of-the-arts, but also help them to comprehend the aspects of a MOT system and the inter-connected aspects. 2) Instead of listing and summarizing individual publications, we categorize the approaches in the key aspects involved in a MOT system. In each aspect, the methods are divided into different groups and each group is discussed in details for the principles, advances and drawbacks. 3) We provide some potential directions with insights for MOT, which are still open issues and need more research efforts. This would be helpful for researchers to identify further interesting problems.
@cite_40: This paper presents to the best of our knowledge the first end-to-end object tracking approach which directly maps from raw sensor input to object tracks in sensor space without requiring any feature engineering or system identification in the form of plant or sensor models. Specifically, our system accepts a stream of raw sensor data at one end and, in real-time, produces an estimate of the entire environment state at the output including even occluded objects. We achieve this by framing the problem as a deep learning task and exploit sequence models in the form of recurrent neural networks to learn a mapping from sensor measurements to object tracks. In particular, we propose a learning method based on a form of input dropout which allows learning in an unsupervised manner, only based on raw, occluded sensor data without access to ground-truth annotations. We demonstrate our approach using a synthetic dataset designed to mimic the task of tracking objects in 2D laser data – as commonly encountered in robotics applications – and show that it learns to track many dynamic objects despite occlusions and the presence of sensor noise.
@cite_15: Recurrent Neural Networks (RNNs), and specifically a variant with Long Short-Term Memory (LSTM), are enjoying renewed interest as a result of successful applications in a wide range of machine learning problems that involve sequential data. However, while LSTMs provide exceptional results in practice, the source of their performance and their limitations remain rather poorly understood. Using character-level language models as an interpretable testbed, we aim to bridge this gap by providing an analysis of their representations, predictions and error types. In particular, our experiments reveal the existence of interpretable cells that keep track of long-range dependencies such as line lengths, quotes and brackets. Moreover, our comparative analysis with finite horizon n-gram models traces the source of the LSTM improvements to long-range structural dependencies. Finally, we provide analysis of the remaining errors and suggests areas for further study.
In multi-object tracking, data association based methods fall into a sub-domain known as the technique, which has shown impressive tracking performance in unconstrained environments. A thorough review can be found in @cite_43 . As evidenced in Section sec:introduction , the local association method has aroused considerable research interests. Especially with the success of recurrent neural networks (RNNs) in computer vision community @cite_15 , RNNs-based methods have witnessed significant advances on MOT problems. Based on the pioneer work introduced by Ondruska and Posner @cite_40 , RNNs-based method quickly sparked significant interest to model the local association, and inspired a number of extensions including @cite_44 @cite_28 @cite_42 . Nevertheless, the RNNs usually comes with high computational and memory demands both during the model training and inference. We here introduce to explicitly enforce locality into the global data association formulation, and introduce a hybrid data association framework that is able to integrate the advantages of both local and global association methods.