1. A method for generating tracking information for an object in a sequence of input images, the method comprising: receiving a first input image from the sequence of input images; receiving a regression model including a covariance matrix represented by a sparse matrix factor, the covariance matrix representing covariance of previously received input images; applying the regression model to the first input image to generate an output prediction representing first tracking information for the object in the first input image; and generating an updated regression model by applying an update to the sparse matrix factor, the update based on the output prediction for the first input image.