and assigned a coefficient such that the total training error of the resulting -stage boosted classifier is minimized. Here is the boosted classifier that has been built up to the previous stage of training and is the weak learner that is being considered for addition to the final classifier. Weighting At each iteration of the training process, a weight is assigned to each sample in the training set equal to the current error on that sample. These weights can be used in the training of the weak learner. For instance, decision trees can be grown which favor the splitting of sets of samples with large weights. Derivation This derivation follows Rojas (2009): Suppose we have a data set where each item has an associated class , and a set of weak classifiers each of which outputs a classification for each item. After the -th iteration our boosted classifier is a linear combination of the weak classifiers of the form: , where the class will be the sign of . At the -th iteration we want to extend this to a better boosted classifier by adding another weak classifier , with another weight : So it remains to determine which weak classifier is the best choice for , and what its weight should be. We define the total error of as the sum of its exponential loss on each data point, given as follows: Letting and for , we have: We can split this summation between those data points that are correctly classified by (so ) and those that are misclassified (so ): Since the only part of the right-hand side of this equation that depends on is , we see that the that minimizes is the one that minimizes [assuming that ], i.e. the weak classifier with the lowest weighted error (with weights ). To determine the desired weight that minimizes with the that we just determined, we differentiate: Setting this to zero and solving for yields: We calculate the weighted error rate of the weak classifier to be , so it follows that: which is the negative logit function multiplied by 0.5. Due to the convexity of as a function of , this new expression for gives the global minimum of the loss function. Note: This derivation only applies when , though it can be a good starting guess in other cases, such as when the weak learner is