1. A training data generation apparatus that generates training data used for creating characteristic expression extraction rules, comprising: a training data candidate generation unit that assigns labels to labeled and unlabeled training data candidates by referring to an incomplete ground truth corpus, a training data candidate clustering unit that clusters a plurality of training data candidates including annotation errors and assigned labels indicating annotation classes based on feature values containing respective context information, and a training data generation unit that generates training data used for creating characteristic expression extraction rules, by referring to each cluster obtained using the clustering results, obtains the distribution of the labels of the training data candidates within the cluster, identifies training data candidates that meet a preset condition based on the obtained distribution, and generates training data using the identified training data candidates, wherein said context information is individual words, modifying or modified phrases in the vicinity of the units that constitute the training data candidates or training data, and wherein a processor operates as at least one of said training data candidate clustering unit and said training data generation unit.