for a loss function L and a neural network f θ(t,i) , with parameters θ at epoch t and mini-batch i. (X i , y i ) is a tuple of inputs and labels in the i-th mini-batch. TSE is correlated with the converged performance of the network and can be used to rank networks early in training to yield substantial compute savings. Given a sufficient number of networks (5-10) that are trained to convergence, a linear regression of the form