1. A method, in a data processing system, for evaluating the performance of a text analysis engine, the method comprising: receiving a plurality of pre-annotated reference documents; receiving a set of annotation types associated with the pre-annotated reference documents; analyzing annotation contexts of reference annotations in the plurality of pre-annotated reference documents using the set of annotation types; identifying similar annotation contexts between the reference annotations and the set of annotation types; responsive to identifying the similar annotation contexts, clustering the similar annotation contexts thereby forming a plurality of reference annotation clusters; computing a set of reference content heterogeneity scores based on the number of reference annotation clusters for each annotation type in the set of annotation types; computing an integral reference content rate for the set of annotation types; and outputting the integral reference content rate to a user.
1. A method, in a data processing system, for evaluating the performance of a text analysis engine, the method comprising: receiving a plurality of pre-annotated reference documents; receiving a set of annotation types associated with the pre-annotated reference documents; analyzing annotation contexts of reference annotations in the plurality of pre-annotated reference documents using the set of annotation types; identifying similar annotation contexts between the reference annotations and the set of annotation types; responsive to identifying the similar annotation contexts, clustering the similar annotation contexts thereby forming a plurality of reference annotation clusters; computing a set of reference content heterogeneity scores based on the number of reference annotation clusters for each annotation type in the set of annotation types; computing an integral reference content rate for the set of annotation types; and outputting the integral reference content rate to a user. 3. The method of claim 1 , wherein clustering the similar annotation contexts groups reference annotations into one or more clusters is based on a similarity of the context of the similar annotation contexts.