8. A system for monitoring a labeler of the speech data, the system comprising: a processor; a memory storing instructions for controlling the processor to perform a method comprising: presenting a test utterance to the labeler; generating a determination indicating whether the labeler correctly labeled the test utterance as one call type from a list of call types; and performing at least one of revising the annotation guide, retraining the labeler, and altering the test utterance based on the determination.
8. A system for monitoring a labeler of the speech data, the system comprising: a processor; a memory storing instructions for controlling the processor to perform a method comprising: presenting a test utterance to the labeler; generating a determination indicating whether the labeler correctly labeled the test utterance as one call type from a list of call types; and performing at least one of revising the annotation guide, retraining the labeler, and altering the test utterance based on the determination. 13. The system of claim 8 , wherein if the test utterance is altered, the instructions further comprise presenting another test utterance related to a different call type.