1. A method comprising: maintaining a model at an online system, the model receiving a plurality of features and generating a response based on the received features; determining a set of new features available for the model based on data maintained by the online system; determining a superset of features for the model, the superset of features comprising the plurality of features used by the model and the new features available for the model; determining measures of feature impact for one or more of the features in the superset of features, the measure of feature impact associated with a feature based at least in part on an effect of the feature on an accuracy of the model; generating a plurality of groups of features from the set of features based at least in part on the determined measures of feature impact, each group of features including one or more features differing from features included in the other groups of the plurality of groups; training the model using each of the plurality of groups of features, each group of features resulting in a performance metric for the model; and presenting information describing the performance metric of the model for each of the groups of features.