15. A non-transitory computer-readable storage medium storing instructions for controlling a computing device to monitor a labeler of the speech data where an annotation guide is used to label utterances in the speech data with a call type, the instructions comprising: presenting a test utterance to the labeler; generating a determination indicating whether the labeler correctly labeled the test utterance as one call type from a list of call types; and based on the determination, performing at least one of revising the annotation guide, retraining the labeler, and altering the test utterance.
17. The non-transitory computer-readable storage medium of claim 15 , wherein presenting the test utterance to the labeler further comprises selecting the test utterance from a group of labeled utterances.