1. A computer implemented method of automatically mapping a given annotator to an aggregate of given annotators comprising: providing a plurality of given annotators; providing aggregates of the plurality of annotators; providing an original document corpus; applying a subject annotator to the original document corpus to generate a corpus of subject annotated documents; generating a corpus of pre-annotated documents that contain annotations; sorting the annotations contained in the pre-annotated documents, to generate good candidate annotations; mapping the subject annotator.