15. A computer readable medium storing instructions for controlling a computing device to monitor labelers of the speech data where an annotation guide is used to label utterances in the speech data with a call type, the instructions comprising: presenting via a processor a test utterance to a labeler; receiving input from the labeler that selects a particular call type from a list of call types; determining via the processor if the labeler labeled the test utterance correctly; and based on the determining step, performing at least one of: revising the annotation guide, retraining the labeler or altering the test utterance.
16. The computer readable medium of claim 15 , wherein presenting a test utterance to a labeler further comprises selecting the test utterance from a group of labeled utterances.