1. In a system that uses annotated speech data, a method for annotating speech data by processing a portion of unannotated speech data with at least one model, the processing comprising: evaluating a performance of at least one model with respect to each utterance in the portion of unannotated speech data using a criterion; creating an annotation list that includes utterances that do not satisfy the criterion by: using system deficiencies in combination with the criterion to identify utterances to be included in the annotation list; and using previously annotated speech data in combination with the criterion or the system deficiencies to identify utterances to be included in the annotation list; identifying an order in which the utterances on the annotation list are to be annotated; generating via a processor a label for a particular utterance; and including the particular utterance in the annotation list if the label does not match an existing label of the particular utterance.