Tracking multiple people across multiple cameras is an open problem. It is typically divided into two tasks: (i) single-camera tracking (SCT) - identify trajectories in the same scene, and (ii) inter-camera tracking (ICT) - identify trajectories across cameras for real surveillance scenes. Many methods cater to SCT, while ICT still remains a challenge. In this paper, we present a feature aggregation architecture called Composite Appearance Network (CAN) to address the above problem. The key structure of this architecture is called EvalNet that pays attention to each feature vector and learns to weight them based on gradients it receives for the overall template for optimal re-identification performance. We demonstrate the efficiency of our approach with experiments on the challenging multi-camera tracking dataset, DukeMTMC. We also survey existing tracking measures and present an online error metric called "Inference Error" (IE) that provides a better estimate of tracking re-identification error, by treating SCT and ICT errors uniformly.
@cite_21: This paper presents a novel approach to feature aggregation for template set based face recognition by incorporating metadata regarding face images to evaluate the representativeness of a feature in the template. We propose using orthogonal data like yaw, pitch, face size, etc. to augment the capacity of deep neural networks to find stronger correlations between the relative quality of the face image in the set with the match performance. The approach presented employs a siamese architecture for training on features and metadata generated using other state-of-the-art CNNs and learns an effective feature fusion strategy for producing optimal face verification performance. We obtain substantial improvements in TAR of over 1.5 at 10^-4 FAR as compared to traditional pooling approaches and illustrate the efficacy of the quality assessment made by the network on the two challenging datasets IJB-A and IARPA Janus CS4.
The framework proposed in this paper is inspired from @cite_21 . The authors propose set based feature aggregation network (FAN) for the face verification problem. By generating representative template features using metadata like yaw, pitch and face size, their system could outperform traditional pooling approaches. Instead of building a siamese network, we focus on learning the pooled feature for gallery template (trajectory) while coupling the probe metadata beside gallery metadata. This way, we derive each trajectory's representation by considering the impact or influence of the probe detection in the context of data association.