8. A system for monitoring labelers of the speech data, the system comprising: a processor; a module controlling the processor to present a test utterance to a labeler; a module controlling the processor to receive input from the labeler that selects a particular call type from a list of call types; a module controlling the processor to determine if the labeler labeled the test utterance correctly; and a module controlling the processor, based on the determination, to perform at least one of: revising the annotation guide, retraining the labeler or altering the test utterance.
9. The system of claim 8 , wherein the module controlling the processor to present a test utterance to a labeler further selects the test utterance from a group of labeled utterances.