35. A process according to claim 30 in step E wherein the step of modifying the extracted and represented features from each document in the selected set is performed using a plurality of different methods.
30. A system to estimate performance of a model to filter documents according to quality comprising: A) a storage device; B) at least one processor to implement the following steps: C) obtaining from the storage device a set of documents labeled according to content quality; D) extracting and representing features from the set of documents; E) modifying the extracted represented features; F) constructing models to estimate performance for labeling documents based on content quality using pattern recognition algorithms stored in the storage device consisting of the following steps; 1) dividing the set of documents into N subsets such that the union of all the subsets is the set of documents; 2) processing each of the N subsets comprising the following steps: a) defining a first set of documents, a second set of documents, and a third set of documents in the subset; b) choosing at least one pattern recognition algorithm; i) instantiating a set of parameters for the pattern recognition algorithm; a′) training the pattern recognition algorithm to build a model using the first set of documents and the parameter set; b′) applying the model to the second set of documents to obtain labels and scores for each document; c′) evaluating the labels and scores; d′) storing the evaluation measure, set of parameters, and current pattern recognition algorithm; and e′) repeating step i) until all appropriate sets of parameters for the pattern recognition algorithm have been applied; and c) repeating step 2b) until all pattern recognition algorithm have been applied; d) aggregating the evaluation measures, the pattern recognition algorithms with the set of parameters; e) selecting the parameter set and pattern recognition algorithm with an aggregate evaluation measure that meets a selection criteria; f) applying the parameter set and the pattern recognition algorithm identified from step e) to the union of the first and second subset to form an intermediate final model; g) applying the intermediate final model to the third set of documents to obtain labels and scores for each document; h) evaluating the labels and scores; i) storing the intermediate evaluation measure; j) repeating step 2 until all subsets have been applied; and 3) aggregating the intermediate evaluation measures to form an aggregate evaluation measure; and 4) displaying the aggregate evaluation measure as the estimated performance.