1. A method comprising: distributing a parameterized model to each worker of a plurality of workers, the plurality of workers being arranged in a hierarchy of workers, the parameterized model including a plurality of feature functions representing a plurality of features and corresponding model parameters; processing a portion of training data at each worker of the plurality of workers according to the parameterized model, the training data including a number of training examples, to calculate updates to model parameters; for each worker at a lowest level of the hierarchy of workers, sending the calculated updates to a next higher level worker; for each other worker in the hierarchy of workers, combining updates of the respective worker with updates received from one or more lower level workers; collecting all updates from the workers at a master to generate real updates to the model parameters; generating an updated model using the real updates to the model parameters; and distributing the updated model to each worker when performing a next iteration.