11. A distributed data annotation server system, comprising: a processor; and a memory configured to store a data annotation application; wherein the data annotation application configures the processor to: obtain a set of source data, where a piece of source data in the set of source data comprises at least one identifying feature; determine a training data set representative of the set of source data, where each piece of source data in the training data set comprises source data metadata describing the ground truth for the piece of source data, where the ground truth for a piece of source data describes the features contained in the piece of source data and a correct label associated with each feature; obtain sets of annotations from a set of annotators for a portion of the training data set, where an annotation identifies one or more features within a piece of source data in the training data set; for each annotator: determine annotator recall metadata based on the set of annotations provided by the annotator for the training data set, where the annotator recall metadata comprises a measure of the number of features within a piece of source data identified with a label in the set of annotations by the annotator; and determine annotator precision metadata based on the set of annotations provided by the annotator for the training data set, where the annotator precision metadata comprises a measure of the number of correct annotations associated with each piece of source data based on the ground truth for each piece of source data; and group the annotators into annotator groups based on the annotator recall metadata and the annotator precision metadata.
12. The system of claim 11 , wherein the data annotation application further configures the processor to generate an annotation task comprising a portion of the set of source data, where the annotation task configures an annotator to annotate one or more features within the set of source data.