8. A non-transitory computer-readable medium storing instructions for controlling a computing device to collect speech data to annotate speech data by processing a portion of unannotated speech data with the at least one model, the instructions comprising: evaluating a performance of the at least one model with respect to each utterance in the portion of unannotated speech data using a criterion; creating an annotation list that includes utterances that do not satisfy the criterion by: using system deficiencies in combination with the criterion to identify utterances to be included in the annotation list; using previously annotated speech data in combination with the criterion or the system deficiencies to identify utterances to be included in the annotation list; identifying an order in which the utterances on the annotation list are to be annotated; generating via a processor a label for a particular utterance; and including the particular utterance in the annotation list if the label does not match an existing label of the particular utterance.
8. A non-transitory computer-readable medium storing instructions for controlling a computing device to collect speech data to annotate speech data by processing a portion of unannotated speech data with the at least one model, the instructions comprising: evaluating a performance of the at least one model with respect to each utterance in the portion of unannotated speech data using a criterion; creating an annotation list that includes utterances that do not satisfy the criterion by: using system deficiencies in combination with the criterion to identify utterances to be included in the annotation list; using previously annotated speech data in combination with the criterion or the system deficiencies to identify utterances to be included in the annotation list; identifying an order in which the utterances on the annotation list are to be annotated; generating via a processor a label for a particular utterance; and including the particular utterance in the annotation list if the label does not match an existing label of the particular utterance. 10. The non-transitory computer-readable medium of claim 8 , wherein each utterance in the portion of the speech data is evaluated using a criterion further comprises at least one of: assigning each utterance in the portion of the speech data a confidence score, wherein utterances having a confidence score below a threshold confidence score are included in the annotation list; evaluating a dialog context of each utterance in the portion of the speech data; and evaluating a feature of each utterance in the portion of the speech data.