1. A method comprising: determining, by a computing device, a plurality of face tracks for detected face samples in a video; receiving, by the computing device, a set of labels for a portion of the plurality of face tracks from a set of users to form a set of labeled face tracks, wherein a label in the set of labels identifies an identity for a face track in the set of labeled face tracks; extracting, by the computing device, a first set of features for a first plurality of face samples in an unlabeled face track in the plurality of face tracks, wherein an identity for the unlabeled face track is not known; correlating, by the computing device, the extracted first set of features for the unlabeled face track to a second set of features from a second plurality of face samples in the set of labeled face tracks; generating, by the computing device, feature weights for the first set of features based on a set of confidence scores for the second set of features using a weighting function that magnifies feature weights for second features from the second plurality of face samples in the second set of features with higher confidence scores more than feature weights for second features from the second plurality of face samples in the second set of features with lower confidence scores in a non-linear manner, wherein a confidence score in the set of confidence scores for a feature in the second plurality of features is based on a quality of a face track in the set of labeled face tracks that includes the feature in the second plurality of features; and using, by the computing device, the generated feature weights to determine a label for the unlabeled face track by applying the generated feature weights to the second set of features.