Deep Meta Learning for Real-Time Target-Aware Visual Tracking
Janghoon Choi Junseok Kwon Kyoung Mu Lee
cs.CV







Deep Meta Learning for Real-Time Target-Aware Visual Tracking
    Janghoon Choi

	ASRI, Department of ECE, 
Seoul National University

	ultio791@snu.ac.kr
	
	
	
	
	
	Junseok Kwon

	School of CSE, 
Chung-Ang Univeristy

	jskwon@cau.ac.kr
	Kyoung Mu Lee

	ASRI, Department of ECE, 
Seoul National University

	kyoungmu@snu.ac.kr

    December 30, 2023
=========================================================================================================================================================================================================================================================================

empty



	In this paper, we propose a novel on-line visual tracking framework based on the Siamese matching network and meta-learner network, which run at real-time speeds. Conventional deep convolutional feature-based discriminative visual tracking algorithms require continuous re-training of classifiers or correlation filters, which involve solving complex optimization tasks to adapt to the new appearance of a target object. To alleviate this complex process, our proposed algorithm incorporates and utilizes a meta-learner network to provide the matching network with new appearance information of the target objects by adding target-aware feature space. The parameters for the target-specific feature space are provided instantly from a single forward-pass of the meta-learner network. By eliminating the necessity of continuously solving complex optimization tasks in the course of tracking, experimental results demonstrate that our algorithm performs at a real-time speed while maintaining competitive performance among other state-of-the-art tracking algorithms.






§ INTRODUCTION


Visual object tracking is one of the fundamental and practical problems among the fields of computer vision research, and it has seen applications in automated surveillance, image stabilization, robotics and more. Given the initial bounding box annotation of an object, visual tracking algorithms aim to track the specified object throughout the subsequent part of the video without losing the object under various circumstances such as illumination change, blur, deformation, fast motion, and occlusion.

Recently, with the increasing use of deep learning and convolutional neural networks (CNN) <cit.> in computer vision applications for their rich representation power and generalization capabilities  <cit.>, there have been numerous studies on utilizing the rich and general feature representation of the CNNs for visual tracking task <cit.>. Most algorithms incorporate the deep convolutional features used in object recognition systems <cit.>. On top of these feature representations, additional classifiers or correlation filters are trained for on-line adaptation to the target object <cit.>.



While these methods were successful in obtaining high performance metrics in well-known benchmarks and datasets <cit.> using deep representations, the majority of these algorithms were not designed as an integrated structure, where two different systems (i.e. deep feature network system and target classifier systems) are built and trained separately, not being closely associated. This causes several problems when the framework is naively applied to the problem of visual tracking, where the classifier system is in constant need of being updated in order to adapt to the appearance changes of the target object, while the number of positive samples are highly limited.

Since an update operation requires solving complex optimization problems for a given objective function using methods such as stochastic gradient descent (SGD) <cit.>, Lagrange multipliers <cit.>, and ridge regression <cit.>, most tracking algorithms with deep representations run at low speeds under 20 fps, thus making real-time applications unrealizable. Moreover, since the updates are often achieved by utilizing a handful of target appearance templates obtained in the course of tracking, while this strategy is inevitable, classifiers are prone to overfitting and losing generalization capabilities due to insufficient positive training samples. To deal with this prevalent overfitting problem, most algorithms incorporate a hand-crafted regularization term with a training hyper-parameter tuning scheme to achieve better results.

Our approach tackles the aforementioned problems by building a visual tracking system incorporating a Siamese matching network for target search and a meta-learner network for adaptive feature space update. We use a fully-convolutional Siamese network structure analogous to <cit.> for searching the target object in a given frame, where target search can be done fast and efficiently using the cross-correlation operations between feature maps. For the meta-learner network, we propose a parameter prediction network inspired by recent advances in the meta learning methodology for few-shot learning problems <cit.>.

The proposed meta-learner network is trained to provide the matching network with additional convolutional kernels and channel attention information so that the feature space of the matching network can be modified adaptively to adopt new appearance templates obtained in the course of tracking without overfitting. The meta-learner network only sees the gradients from the last layer of the matching network, given new training samples for the appearance. We also employ a novel training scheme for the meta-learner network to maintain the generalization capability of the feature space by preventing the meta-learner network from generating new parameters that cause overfitting of the matching network. By incorporating our meta-learner network, the target-specific feature space can be constructed instantly with a single forward pass without any iterative computation for optimization and is free from innate overfitting, improving the performance of the tracking algorithm. Fig.<ref> illustrates the motivation of the proposed visual tracking algorithm. We show the effectiveness of our method by showing consistent performance gains in 5 different visual tracking datasets <cit.> while maintaining a real-time speed of 48 fps.






§ RELATED WORK


General visual tracking approaches: 
Conventional tracking algorithms can be largely grouped into two approaches. One approach builds a generative appearance model of the target based on previously observed examples. This generative model can be used to find the target in the upcoming frames by finding the region that can be best described by the model, where sparse representation and linear subspace representation is often utilized <cit.>. The other approach aims to build a discriminative classifier to distinguish the target region from the background region. This discriminative classifier can be used to find the target region in the upcoming frames by solving a binary classification problem <cit.>. Recently, correlation filters have gained great popularity among the visual tracking methods since the seminal works of <cit.> and <cit.> due to their simplicity and computational efficiency in the Fourier frequency domain. Many new approaches have been proposed based on the correlation filter learning framework such as color attribute features <cit.>, using multi-resolution feature maps <cit.>, accurate scale estimation <cit.>, spatial regularization <cit.>, and factorized convolution operators <cit.>. 


Visual tracking methods using deep representations: 
With the growing popularity of the application of deep convolutional networks to a wide range of computer vision tasks, many novel visual tracking algorithms make use of the powerful representation capabilities of the convolutional neural networks (CNN). Starting with <cit.>, where the encoder representation of the denoising autoencoder was used, <cit.> used feature representations of the VGG-M network <cit.> and  <cit.> also used VGG feature maps. Many correlation filter-based tracking algorithms also utilize the powerful representation capacity of the CNN by training the correlation filters on the feature maps of the network. Recent approaches include hierarchical correlation filters <cit.>, adaptive hedging of correlation filters <cit.>, continuous convolutional operators <cit.>, sequential training of features <cit.>, and spatial regularization <cit.>. Other than correlation filter-based algorithms, approaches to design an end-to-end framework for visual tracking have recently emerged. They employ the two-flow Siamese architecture networks commonly used in stereo matching <cit.> and patch-matching <cit.> problems. <cit.> and <cit.> trained a Siamese network to learn the two-patch similarity function that shares the convolutional representation. <cit.> proposed a more end-to-end approach to visual tracking where the Siamese network can localize an exemplar patch inside a search patch. They use a  fully convolutional architecture that adopts a cross-correlation layer which significantly lowers the computational complexity. Based on the framework of <cit.>, recent approaches incorporate triplet loss <cit.>, region proposal networks <cit.>, distractor-aware features for suppressing semantic distractors <cit.> and two-fold Siamese networks for semantic and appearance features <cit.>. 


Meta learning methods for few-shot image recognition task and visual tracking: 
There are recent approaches for learning to classify from a few given examples using meta learning methodologies <cit.>. In <cit.>, authors proposed a network architecture that employs characteristics of non-parametric nearest-neighbor models to solve N-way, k-shot learning tasks, where a small support set is given. <cit.> made use of a pre-trained network as a good initialization and then trained the meta-learner to effectively fine-tune the network based on few given examples. In <cit.>, a two-level structure of meta-learner and base-learner both equipped with fast and slow weights was used. The meta-learner acquires the meta information from the base-learner in the form of loss gradients, and then provides the based-learner with fast parameterization while preserving generalization capabilities. Recently, <cit.> proposed a meta-learner based optimizer analogous to <cit.> for the visual tracking task, where they chose <cit.> and <cit.> as the baseline algorithms to show the effectiveness of their update step, decreasing the number of training iterations thus improving the speed of the baseline methods. Whereas the aim of our method is to update the network using the meta-learner with a single iteration at real-time speeds, providing the network with new adaptive kernels and feature space representation without overfitting, which can be achieved by our regularizing training scheme.




§ TRACKING WITH META-LEARNER


In the following subsections, we first provide an overview of our proposed visual tracking framework, where a brief explanation of our framework and the visual tracking procedure are given. Then we describe the implementation and training details for the components of our framework. 




 §.§ Overview of Proposed Method
 



  §.§.§ Components

Our framework is largely composed of two components, a matching network and a meta-learner network. The matching network is a fully-convolutional Siamese network that takes two images as inputs where x is denoted as an image patch of the target and z is an image patch of the larger context area that contains the target. The matching network takes these inputs, extracts the feature maps using the N-layer feature extractor CNN network ϕ_w(·), and produces the final response map f_w(x,z) by cross-correlation operation between the feature maps. This process can be expressed as follows,

    f_w(x,z)=ϕ_w(x) ∗ϕ_w(z),

where ∗ represents the cross-correlation operator between two feature maps and w={w_1,w_2,...,w_N} represents the set of trained kernel weights for each layer of the feature extractor CNN. To train the feature extractor CNN, we minimize a differentiable loss function given as ℓ(f_w(x,z),y), where the loss function measures the inaccuracy in predictions of f_w, given y as the ground-truth response map. 

The meta-learner network provides the matching network with target-specific weights given an image patch of the target x with context patches z_δ={z_1,...,z_M} previously obtained and cropped around the target's vicinity. To adapt the weights to the target patch, we use the averaged negative gradient δ of the loss function for the last layer of the matching network taken as, 

    δ = ∑_i=1^M - 1/M∂ℓ(f_w(x,z_i),ỹ_i)/∂ w_N,

where ỹ_i is the generated binary response map assuming the target is located at the correct position inside the context patch z_i. 
The meta-learner network is designed based on the fact that the characteristic of δ is empirically different according to a target. 
Then, given δ as an input, the meta-learner network g_θ(·) can generate target-specific weights w^target corresponding to the input as, 

    w^target = g_θ(δ),

where θ is the parameter for the meta-learner network. The new weights are used to update the matching network's original weights as in,

    f_w^adapt(x,z)=ϕ_w^adapt(x) ∗ϕ_w^adapt(z),

where w^adapt={w_1,w_2,...,[w_N,w^target]}, concatenating w^target to w_N of the last layer for feature extraction. The meta-learner network also generates channel-wise sigmoid attention weights for each channel of the feature map to further adjust the feature representation space where the weights can be applied by channel-wise multiplication.
Fig.<ref> shows an overview of the proposed method. 








  §.§.§ Tracking algorithm

Tracking is performed in a straightforward and simplistic manner to ensure fast performance. Given a target patch x and its previous state, a context image z in a new frame can be cropped based on the previous state. Processing both images through the matching network, the estimated response map ŷ = f_w^adapt(x,z) is obtained. 
The new position of the target can be found by finding the maximum position in the response map ŷ⊗ h, where ⊗ is an element-wise multiplication operator and h is a cosine window function for penalizing large displacements. Scale variation of the target can be covered by using multiple size crops of z matched with x. Scale changes are also penalized and damped by a constant to ensure smooth changes of target size over time. 

During the course of tracking, we keep a memory of the context images as z_mem = {z_1,...,z_K} along with the corresponding estimated response maps used for tracking ŷ_mem = {ŷ_1, ..., ŷ_K }. We store a context image z to the memory only if it is considered to be confident, where the maximum response value in the corresponding map ŷ is over a certain threshold τ. To update the appearance model of the target, we choose M samples from this memory under the minimum entropy criterion on ŷ_mem as in <cit.> without replacement. This criterion is used to avoid ambiguous response maps where false positive samples may exist in the corresponding context image. Finding the response map with the minimum entropy can be defined as,

    _ŷ_i ∈ŷ_mem -∑_p∈𝒫ρ (ŷ_i[p]) log (ρ (ŷ_i[p])),

where p corresponds to a position in a set of all possible positions 𝒫 in the response map and ρ(·) is the normalization function. Using the chosen M appearance samples z_δ , target-adaptive weights w^target are obtained using the meta-learner network as in (<ref>) and (<ref>), and then the matching network is updated as in (<ref>), and it is used to track the object in subsequent frames. Since updating the model too frequently is unnecessary and cumbersome for the performance, we only update the model every T frames as in other algorithms <cit.>. The overall tracking process is described in Algorithm <ref>.








 §.§ Network Implementation and Training
 




  §.§.§ Matching Network


The matching network consists of a shared feature extractor CNN ϕ(·), channel-wise attention step, feature ℓ^2-normalization step, and cross-correlation step. For feature extraction, we use a CNN with 5 convolutional layers and 2 pooling layers of kernel size 3 and stride 2 are applied after the first two convolutional layers. Batch normalization layer is inserted after each convolutional layer. The overall structure of the CNN is analogous to <cit.>, where kernel size and input/output dimensions for each layer are w_1:11×11×3×128, w_2:5×5×128×256, w_3:3×3×256×384, w_4:3×3×384×256, w_5:1×1×256×192. For inputs, we use a RGB image of size 127×127×3 for x and a RGB image of size 255×255×3 for z, and the matching network produces a response map of size 17×17.

To train the matching network, we used ILSVRC 2015 <cit.> object detection from video dataset with additional training data from ILSVRC 2017 dataset, which contains objects of 30 classes in 4000 videos in the training set and 1314 videos in the validation set, with a total of 11566 independent object trajectories. Each frame of the video is annotated with bounding box notations of objects appearing in the video. We only used videos in the training set to train the matching network. At training time, pairs of (x,z) are randomly sampled from an object trajectory in a chosen video clip. Then, a ground-truth response map y ∈{-1,+1}^17 × 17 is generated where the value is +1 at the position of the target and -1 otherwise.  For the loss function ℓ(f_w(x,z),y), we use the logistic loss function defined as,

    ℓ(f_w(x,z),y) = 
       1/|𝒫|∑_p∈𝒫ζ(y[p]) ·log(1 + exp(-f_w(x,z)[p] · y[p])),

where p represents a position in the set of every possible positions 𝒫 in the response map and ζ(y[p]) is a weighting function for alleviating label imbalance. The loss function is optimized with Adam  <cit.> optimizer with the learning rate of 10^-4 using batch size of 8, and run for 95000 iterations. 




  §.§.§ Meta-Learner Network




We then train the meta-learner network subsequent to pre-training the matching network. The meta-learner network g_θ(·) consists of 3 fully-connected layers with 2 intermediate layers of 512 units. Each intermediate layer is followed by a dropout layer with the keep probability of 0.7 when training. For input, gradient δ of size 1×1×256×192 is used and output w^target of size 1×1×256×32 is generated. These new kernels are used to update the weights of the matching network by concatenating w^target to the kernels w_5 of the last layer of the Siamese matching network to provide the additional feature space needed for updates, resulting in new kernels [w_5,w^target] of size 1×1×256×(192+32). 

To train the meta-learner network, we use 1314 videos in the validation set of the ILSVRC video dataset. The training process is described hereafter. First, an anchor target image x is randomly sampled from an object trajectory. Then, M' context patches are randomly sampled from the same object's trajectory as in z_reg={z_1,...,z_M'} where M'≥ M. Then M patches are chosen from z_reg to form z_δ, where we can perform matching these samples with the target image x to obtain gradient δ by (<ref>) using generated binary response map ỹ_i, assuming the target is located at the center of z_i. Standard data augmentation techniques (e.g. horizontal flip, noise, Gaussian blur, translation) are applied when sampling z_i. We can train the meta-learner network g_θ(δ) by minimizing the loss function with respect to parameter θ:

    _θ∑_z_i ∈z_regℓ(f_w^adapt(x,z_i),y), where
    w^adapt={w_1,w_2,...,[w_N,g_θ(δ)]}.





Training the meta-learner network to generate new weights w^target=g_θ(δ) that only fit examples in z_δ (i.e. M'=M ) can lead the meta-learner network to generate weights that will make the matching network overfit to samples in z_δ. To prevent this overfitting problem, a regularization scheme is needed when training. For natural regularization, M'=2M is chosen so that the weights can fit a larger set of examples z_reg instead of a smaller set z_δ. This encourages much better generalization properties for the matching network when tracking. For the experiments, M=8 and M'=16 are used and Adam optimizer with the learning rate of 10^-4 with batches of 8 videos are used. Training is performed for 11000 iterations.
Fig.<ref> shows the training scheme of the meta-learner network. 





§ EXPERIMENTAL RESULTS







 §.§ Evaluation Environment

Our algorithm was implemented in Python using TensorFlow 1.8.0 <cit.> library and executed on a system with Intel Core i7-4790K 4GHz CPU with 32GB of RAM with GeForce GTX TITAN X (Maxwell) GPU with 12GB of VRAM. The algorithm ran at an average of 48.1 fps on 100 videos in the OTB-2015 <cit.> dataset. We considered 3 scale variations of [1.00, 1/1.035, 1.035] to adapt to the scale change of the target, where changes in scale are penalized by a constant of 0.97 and damped by a constant of 0.59. Cosine window h was applied with the penalization factor of 0.25. The meta-learner network updated the weights every T=30 frames, and threshold of τ=0.5 was used for choosing confident samples. All parameters were fixed during the entire evaluation process for all datasets.




 §.§ Experiments and Analysis

Object Tracking Benchmark (OTB) <cit.> is a visual tracking benchmark that is widely used to evaluate the performance of a visual tracking algorithm. The dataset contains a total of 100 sequences and each is annotated frame-by-frame with bounding boxes and 11 challenge attributes. OTB-2013 dataset contains 51 sequences and the OTB-2015 dataset contains all 100 sequences of the OTB dataset. As evaluation metric, we used OPE success rate evaluation metric that compares the predicted bounding boxes with the ground truth bounding boxes to obtain intersection over union (IoU) scores, and measure the proportion of predictions having larger score than a given varying threshold score value. The final score is calculated by measuring the area-under-curve (AUC) for each tracker. Large-scale Single Object Tracking (LaSOT) <cit.> dataset is a recently introduced large-scale visual tracking dataset containing 1400 sequences with average length of 2512 frames (83 secs) and minimum of 1000 frames per sequence, with the total of 3.52 million frames where every frame is annotated with a bounding box annotation. It contains 70 object categories with each containing 20 sequences. Compared to OTB, LaSOT contains 14 times more sequences and 59 times the total number of frames, with more various object categories.  For evaluation protocols, Protocol I employs all 1400 sequences for evaluation and Protocol II uses the testing subset of 280 videos, where AUC of success plot is used for the performance metric for both protocols. 

We also perform internal comparisons on TC-128 <cit.>, UAV20L <cit.> and VOT-2016 <cit.> datasets to show the effectiveness of our meta-learner update scheme. VOT-2016 is the dataset used for the VOT Challenge <cit.> and it contains a total of 60 videos with bounding box annotations. Baseline experiment of the VOT dataset performs re-initialization of the tracker when it misses the target, while unsupervised experiment simply lets the tracker run from the first frame to the end. Temple Color-128 (TC-128) dataset <cit.> contains 128 real-world color videos annotated with bounding boxes, with 11 challenge factors. UAV20L <cit.> dataset contains 20 video sequences with an average length of 2933.5 frames, where some sequences have targets leaving the video frame (out-of-view). No explicit failure detection or re-detection scheme was used for all experiments.






  §.§.§ Quantitative Analysis


Effect of meta-learner network:  
We performed an internal comparison between the proposed tracker (MLT) and the baseline trackers MLT-mt and MLT-mt+ft, where MLT-mt is a variant that has only the matching network with fixed weights without the meta-learner network, and MLT-mt+ft performs on-line finetuning on  (kernel w_5) with training examples obtained while tracking. For fair comparison, the baseline trackers are pretrained on the whole ImageNet video detection dataset including the validation set, with the last convolutional layer of kernel size 1×1×256×224. For the MLT-mt+ft method, we finetune the matching network every 50 frames for 30 iterations using the Adam optimizer with the learning rate of 10^-3. As shown in Table <ref>, the meta-learner network improves the performance of the baseline matching network and produces better tracking results. MLT is consistently superior to MLT-mt and MLT-mt+ft in OTB, LaSOT, TC-128, UAV20L and VOT2016 datasets. The results demonstrate that the adaptive weights generated by meta-learner network are effective for inducing the customized feature space for each target and for resulting in accurate visual tracking, showing performance gains in all 5 datasets. Also, results of MLT-mt+ft show that online finetuning without handpicked hyper-parameters and regularization scheme easily results in overfitting to handful of training samples, resulting in lower performance.







Comparison with other trackers:  
We compare our tracking algorithm MLT with 12 tracking algorithms on the OTB and LaSOT datasets, namely, SiamFC <cit.>, StructSiam <cit.>, DSiam <cit.>, CFNet <cit.>, SINT <cit.>, SRDCF <cit.>, PTAV <cit.>, ECO-HC <cit.>, STAPLECA <cit.>, BACF <cit.>, DSST <cit.> and HDT <cit.>, where most are real-time algorithms.
As shown in Table <ref>, MLT achieves a competitive accuracy on OTB datasets compared to other tracking algorithms based on deep representation, and outperforms other algorithms on large-scale LaSOT experiments due to its robust meta-learner update. Especially, we were able to obtain a noticeable performance gain compared to SiamFC and its variants (StructSiam, DSiam and CFNet) on LaSOT where no variant was able to outperform the original SiamFC.

We also separately analyze the performance of MLT with respect to 8 different attributes of OTB videos. 
Each video has a different attribute such as in-plane rotation, out-of-plane rotation, motion blur, low resolution, scale variation, illumination variation, background clutter and occlusion.
Fig. <ref> shows that MLT is robust to low resolution, occlusion and scale variation and is competitive to the other trackers for most of the attributes.
In blurred low resolution images, the appearance of a target is frequently indistinguishable from that of other objects in the background. MLT can distinguish between these appearances by customizing features spaces for each target with the meta-learner network. Also, MLT can learn from negative examples from the background and handle occlusions better than other trackers.




  §.§.§ Qualitative Analysis

Fig.<ref> shows qualitative tracking results produced by SiamFC, SRDCF, HDT, CNN-SVM, DSST, and the proposed algorithm MLT.
All trackers were tested on all videos in the OTB-2015 dataset where tracking results for selected videos are shown in Fig.<ref> due to the limit in the length of the paper.
MLT robustly and accurately tracks the target in spite of several challenging conditions such as occlusion in Box seq., pose variation in Rubik seq., background clutter in Human3 seq., fast motion in Girl2 seq., and scale variation in Car24 seq..  
These qualitative tracking results demonstrate that the proposed MLT successfully exploited the power of the meta-learner network and utilized the adaptive weights customized for each target to improve tracking accuracy, without losing the generalization capabilities.   
For reference, we attach a supplementary video containing more qualitative results on the OTB-2015 dataset.

In addition, Fig.<ref> shows some examples of how the target-specific feature space modifies the response maps, thus demonstrating how the meta-learner can be beneficial in the task of visual tracking. We show example image patches z where the target object fixed at the center of the image, with response maps before and after the target-specific feature space modification. The response maps show that the target-specific weights help the tracker to adapt to various target appearance changes and to locate the target, and are also effective in avoiding false positives by suppressing incorrect responses from distractors in the background.




§ CONCLUSION

In this paper, we proposed a novel visual tracking algorithm based on the target-specific feature space constructed by the deep meta-learner network.
The proposed tracking algorithm adapts to the target appearance by generating the target-specific adaptive weights with the meta-learner network, where the matching network provides the meta-information gradients as a learning signal. Our algorithm aims to customize the feature space to discriminate a specific target appearance from the background in order to accurately track the target without overfitting.
Experimental results demonstrate that our algorithm achieves a noteworthy performance gain in visual tracking by using the proposed meta-learner network, achieving consistent performance gains on 5 tracking datasets including the large-scale tracking dataset LaSOT. 
Quantitatively and qualitatively the algorithm shows a competitive tracking performance on multiple visual tracking datasets with several challenging tracking conditions, compared to other visual tracking algorithms, while running at the real-time speed of 48 fps.



  §.§.§ Acknowledgments

This work was supported by IITP grant funded by the Ministry of Science and ICT of Korea (No.2017-0-01780, The technology development for event recognition/relational reasoning and learning knowledge based system for video understanding).


ieeefullname
	
	10=-1pt
		
		tensorflow
		Martın Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen,
		Craig Citro, Greg S Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, et al.
		Tensorflow: Large-scale machine learning on heterogeneous distributed
		systems.
		arXiv preprint arXiv:1603.04467, 2016.
		
		siamfc
		Luca Bertinetto, Jack Valmadre, João F Henriques, Andrea Vedaldi, and
		Philip HS Torr.
		Fully-convolutional siamese networks for object tracking.
		arXiv preprint arXiv:1606.09549, 2016.
		
		mosse
		David S Bolme, J Ross Beveridge, Bruce A Draper, and Yui Man Lui.
		Visual object tracking using adaptive correlation filters.
		In CVPR, 2010.
		
		vot
		Luka Čehovin, Aleš Leonardis, and Matej Kristan.
		Visual object tracking performance measures revisited.
		IEEE TIP, 25(3):1261–1274, 2016.
		
		ECO
		Martin Danelljan, Goutam Bhat, Fahad Khan, and Michael Felsberg.
		Eco: Efficient convolution operators for tracking.
		In CVPR, 2017.
		
		dsst
		Martin Danelljan, Gustav Häger, Fahad Khan, and Michael Felsberg.
		Accurate scale estimation for robust visual tracking.
		In BMVC, 2014.
		
		deepsrdcf
		Martin Danelljan, Gustav Hager, Fahad Shahbaz Khan, and Michael Felsberg.
		Convolutional features for correlation filter based visual tracking.
		In ICCV Workshop, 2015.
		
		srdcf
		Martin Danelljan, Gustav Hager, Fahad Shahbaz Khan, and Michael Felsberg.
		Learning spatially regularized correlation filters for visual
		tracking.
		In CVPR, 2015.
		
		ccot
		Martin Danelljan, Andreas Robinson, Fahad Shahbaz Khan, and Michael Felsberg.
		Beyond correlation filters: Learning continuous convolution operators
		for visual tracking.
		In ECCV, 2016.
		
		color
		Martin Danelljan, Fahad Shahbaz Khan, Michael Felsberg, and Joost Van de
		Weijer.
		Adaptive color attributes for real-time visual tracking.
		In CVPR, 2014.
		
		siamtriplet
		Xingping Dong and Jianbing Shen.
		Triplet loss in siamese network for object tracking.
		In ECCV, 2018.
		
		LaSOT
		Heng Fan, Liting Lin, Fan Yang, Peng Chu, Ge Deng, Sijia Yu, Hexin Bai, Yong
		Xu, Chunyuan Liao, and Haibin Ling.
		Lasot: A high-quality benchmark for large-scale single object
		tracking.
		arXiv preprint arXiv:/1809.07845, 2018.
		
		ptav
		Heng Fan and Haibin Ling.
		Parallel tracking and verifying: A framework for real-time and high
		accuracy visual tracking.
		In ICCV, 2017.
		
		maml
		Chelsea Finn, Pieter Abbeel, and Sergey Levine.
		Model-agnostic meta-learning for fast adaptation of deep networks.
		In ICML, 2017.
		
		dsiam
		Qing Guo, Wei Feng, Ce Zhou, Rui Huang, Liang Wan, and Song Wang.
		Learning dynamic siamese network for visual object tracking.
		In ICCV, 2017.
		
		matchnet
		Xufeng Han, Thomas Leung, Yangqing Jia, Rahul Sukthankar, and Alexander C Berg.
		Matchnet: Unifying feature and metric learning for patch-based
		matching.
		In CVPR, 2015.
		
		Struck
		Sam Hare, Amir Saffari, and Philip Torr.
		Struck: Structured output tracking with kernels.
		In ICCV, 2011.
		
		sasiam
		Anfeng He, Chong Luo, Xinmei Tian, and Wenjun Zeng.
		A twofold siamese network for real-time object tracking.
		In CVPR, 2018.
		
		100fps
		David Held, Sebastian Thrun, and Silvio Savarese.
		Learning to track at 100 fps with deep regression networks.
		In ECCV, 2016.
		
		KCF
		João F Henriques, Rui Caseiro, Pedro Martins, and Jorge Batista.
		High-speed tracking with kernelized correlation filters.
		IEEE TPAMI, 37(3):583–596, 2015.
		
		cnnsvm
		Seunghoon Hong, Tackgeun You, Suha Kwak, and Bohyung Han.
		Online tracking by learning discriminative saliency map with
		convolutional neural network.
		In ICML, 2015.
		
		Muster
		Zhibin Hong, Zhe Chen, Chaohui Wang, Xue Mei, Danil Prokhorov, and Dacheng Tao.
		Multi-store tracker (muster): A cognitive psychology inspired
		approach to object tracking.
		In CVPR, 2015.
		
		TLD
		Zdenek Kalal, Krystian Mikolajczyk, and Jiri Matas.
		Tracking-learning-detection.
		IEEE TPAMI, 34(7):1409–1422, 2012.
		
		bacf
		Hamed Kiani Galoogahi, Ashton Fagg, and Simon Lucey.
		Learning background-aware correlation filters for visual tracking.
		In ICCV, 2017.
		
		adam
		Diederik Kingma and Jimmy Ba.
		Adam: A method for stochastic optimization.
		ICLR, 2015.
		
		alexnet
		Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton.
		ImageNet classification with deep convolutional neural networks.
		In NIPS, 2012.
		
		lenet
		Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner.
		Gradient-based learning applied to document recognition.
		Proceedings of the IEEE, 86(11):2278–2324, 1998.
		
		siamrpn
		Bo Li, Junjie Yan, Wei Wu, Zheng Zhu, and Xiaolin Hu.
		High performance visual tracking with siamese region proposal
		network.
		In CVPR, 2018.
		
		tc128
		Pengpeng Liang, Erik Blasch, and Haibin Ling.
		Encoding color information for visual tracking: Algorithms and
		benchmark.
		IEEE TIP, 24(12):5630–5644, 2015.
		
		csrdcf
		Alan Lukezic, Tomas Vojir, Luka Cehovin Zajc, Jiri Matas, and Matej Kristan.
		Discriminative correlation filter with channel and spatial
		reliability.
		In CVPR, 2017.
		
		convcorr
		Chao Ma, Jia-Bin Huang, Xiaokang Yang, and Ming-Hsuan Yang.
		Hierarchical convolutional features for visual tracking.
		In CVPR, 2015.
		
		L1
		Xue Mei and Haibin Ling.
		Robust visual tracking using l1 minimization.
		In ICCV, 2009.
		
		uav123
		Matthias Mueller, Neil Smith, and Bernard Ghanem.
		A benchmark and simulator for uav tracking.
		In ECCV, 2016.
		
		cacf
		Matthias Mueller, Neil Smith, and Bernard Ghanem.
		Context-aware correlation filter tracking.
		In CVPR, 2017.
		
		metanet
		Tsendsuren Munkhdalai and Hong Yu.
		Meta networks.
		In ICML, 2017.
		
		mdnet
		Hyeonseob Nam and Bohyung Han.
		Learning multi-domain convolutional neural networks for visual
		tracking.
		In CVPR, 2015.
		
		metatracker
		Eunbyung Park and Alexander C Berg.
		Meta-tracker: Fast and robust online adaptation for visual object
		trackers.
		In ECCV, 2018.
		
		hdt
		Yuankai Qi, Shengping Zhang, Lei Qin, Hongxun Yao, Qingming Huang, Jongwoo Lim,
		and Ming-Hsuan Yang.
		Hedged deep tracking.
		In CVPR, 2016.
		
		metalstm
		Sachin Ravi and Hugo Larochelle.
		Optimization as a model for few-shot learning.
		In ICML, 2016.
		
		frcnn
		Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun.
		Faster r-cnn: Towards real-time object detection with region proposal
		networks.
		In NIPS, 2015.
		
		IVT
		David Ross, Jongwoo Lim, Ruei-Sung Lin, and Ming-Hsuan Yang.
		Incremental learning for robust visual tracking.
		IJCV, 77(1–3):125–141, 2008.
		
		imagenet
		Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma,
		Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al.
		Imagenet large scale visual recognition challenge.
		IJCV, 115(3):211–252, 2015.
		
		vgg
		Karen Simonyan and Andrew Zisserman.
		Very deep convolutional networks for large-scale image recognition.
		arXiv preprint, arXiv:1409.1556, 2014.
		
		crest
		Yibing Song, Chao Ma, Lijun Gong, Jiawei Zhang, Rynson Lau, and Ming-Hsuan
		Yang.
		Crest: Convolutional residual learning for visual tracking.
		In ICCV, 2017.
		
		siamis
		Ran Tao, Efstratios Gavves, and Arnold W M Smeulders.
		Siamese instance search for tracking.
		In CVPR, 2016.
		
		cfnet
		Jack Valmadre, Luca Bertinetto, Joao Henriques, Andrea Vedaldi, and Philip
		H. S. Torr.
		End-to-end representation learning for correlation filter based
		tracking.
		In CVPR, 2017.
		
		matchos
		Oriol Vinyals, Charles Blundell, Tim Lillicrap, Koray Kavukcuoglu, and Daan
		Wierstra.
		Matching networks for one shot learning.
		In NIPS, 2016.
		
		fctrack
		Lijun Wang, Wanli Ouyang, Xiaogang Wang, and Huchuan Lu.
		Visual tracking with fully convolutional networks.
		In ICCV, 2015.
		
		stct
		Lijun Wang, Wanli Ouyang, Xiaogang Wang, and Huchuan Lu.
		Stct: Sequentially training convolutional networks for visual
		tracking.
		In CVPR, 2016.
		
		nipstrack
		Naiyan Wang and Dit-Yan Yeung.
		Learning a deep compact image representation for visual tracking.
		In NIPS, 2013.
		
		otb
		Yi Wu, Jongwoo Lim, and Ming-Hsuan Yang.
		Object tracking benchmark.
		IEEE TPAMI, 37(9):1834–1848, 2015.
		
		mccnn
		Jure Žbontar and Yann LeCun.
		Stereo matching by training a convolutional neural network to compare
		image patches.
		JMLR, 17(1):2287–2318, 2016.
		
		meem
		Jianming Zhang, Shugao Ma, and Stan Sclaroff.
		MEEM: robust tracking via multiple experts using entropy
		minimization.
		In ECCV, 2014.
		
		CST
		Tianzhu Zhang, Adel Bibi, and Bernard Ghanem.
		In defense of sparse tracking: Circulant sparse tracker.
		In CVPR, 2016.
		
		MTT
		T. Zhang, B. Ghanem, S. Liu, and N. Ahuja.
		Robust visual tracking via multi-task sparse learning.
		In CVPR, 2012.
		
		MCPF
		Tianzhu Zhang, Changsheng Xu, and Ming-Hsuan Yang.
		Multi-task correlation particle filter for robust object tracking.
		In CVPR, 2017.
		
		stsiam
		Yunhua Zhang, Lijun Wang, Jinqing Qi, Dong Wang, Mengyang Feng, and Huchuan Lu.
		Structured siamese network for real-time visual tracking.
		In ECCV, 2018.
		
		dasiamrpn
		Zheng Zhu, Qiang Wang, Bo Li, Wei Wu, Junjie Yan, and Weiming Hu.
		Distractor-aware siamese networks for visual object tracking.
		In ECCV, 2018.
		
	


