1. A method for generating tracking information for an object in a sequence of input images, the method comprising: receiving a first input image from the sequence of input images; receiving a regression model including a covariance matrix represented by a sparse matrix factor, the covariance matrix representing covariance of previously received input images; applying the regression model to the first input image to generate an output prediction representing first tracking information for the object in the first input image; and generating an updated regression model by applying an update to the sparse matrix factor, the update based on the output prediction for the first input image, wherein applying the update to the sparse matrix factor comprises an O(n) runtime operation, where n is a number of received images.