12. A computer system comprising: a processor(s) set; and a computer readable storage medium; wherein: the processor set is structured, located, connected and/or programmed to run program instructions stored on the computer readable storage medium; and the program instructions include: program instructions programmed to cause a first natural language processing (NLP) annotator of an NLP system to annotate a corpus, thereby producing an annotated corpus that includes a first set of annotation(s); program instructions programmed to cause a second NLP annotator of the NLP system to annotate the annotated corpus that includes the first set of annotation(s), thereby producing a second set of annotation(s) that annotate the annotated corpus that includes the first set of annotation(s); program instructions programmed to determine, based, at least in part, on the second set of annotation(s), that a first annotation error has occurred in the annotation of the corpus by the first NLP annotator; program instructions programmed to identify a cause of the first annotation error; program instructions programmed to generate a candidate set of annotation correction actions, where each annotation correction action of the set is directed to the identified cause and is adapted to prevent an occurrence of an error similar to the first annotation error by the first NLP annotator; program instructions programmed to select an annotation correction action from the candidate set of annotation correction actions, based, at least in part, on a set of annotation correction confidence characteristics and based, at least in part, on an impact analysis performed against one or more ground truth annotations, the ground truth annotations having been performed by humans; and program instructions programmed to automatically apply the selected annotation correction action to the first NLP annotator; and program instructions programmed to cause the first NLP annotator to annotate a new corpus, thereby producing new annotation(s) based on the applied annotation correction action.
13. The computer system of claim 12 , wherein determining that the first annotation error has occurred is based, at least in part, on metrics collected during the annotation of the corpus by the first NLP annotator.