1. A method of creating a classifier model for a multi-class linear classifier, the classifier model including feature weights for each of a plurality of feature and label combinations, the method comprising iteratively performing the steps of: selecting a plurality of feature weights from the classifier model to update for inclusion in an update set, the selection performed without respect to feature weights selected in any previous iteration, and, in a plurality of iterations, the number of feature weights in the plurality of feature weights being greater than in a previous iteration; determining an update for each of the feature weights included in the update set based on a line-search of a loss function l that incorporates a multi-class L 1 -regularized hinge loss function that is continuous and piece-wise linear, the line-search performed using the following steps for each of those feature weights: identifying, for each of the selected feature weights, points representative of values u resulting in a change in slope of an unregularized loss output function l (f,l u ) (u) or a regularization penalty output function r (f,l u ) (u), the output functions each derived from the loss function l, determining at least one derivative value for an initial point of the output functions and at least one change in derivative value for at least some of the points using a processor, determining a cumulative reduction in loss for at least some of the points using at least one of the determined derivative values or at least one of the changes in derivative values, and determining the update based on a point having a largest cumulative reduction in loss; combining the update for each of the feature weights into an update vector; determining a step size to modify a magnitude of each of the updates; modifying the update vector using the step size; creating an updated classifier model using the modified update vector; and stopping the iteration if the updated classifier model or a recently updated classifier model is determined to be sufficiently accurate.