6. The system of claim 1 , further comprising: a test information obtaining unit to, obtain test information comprising two pieces of correct answer information indicating whether to attach each annotation from the first plurality of annotations and the second plurality of annotations to a test document group used to test the annotator, and two scores indicating a third certainty of each of the annotations attached to the test document group; a calculation target information obtaining unit to, obtain calculation target information comprising two scores, each of the two scores indicating a fourth certainty of each annotation from the first plurality of annotations and the second plurality of annotations attached to a calculation target document group used to calculate a correlation between the annotations; and wherein the computing unit, computes the estimated value of the correlation using the test information and the calculation target information.
8. The system of claim 6 , wherein the correct answer information is obtained by evaluation of each test document of the test document group, the correct answer information indicating whether to attach the annotations.