1. A method, in a data processing system comprising a processor and a memory, for configuring a ranker array of a question and answer (QA) system, comprising: generating, by the data processing system, an initial configuration of a ranker array of the QA system, wherein the ranker array comprises a plurality of rankers, and wherein each ranker in the ranker array applies a statistical classification function to candidate answers generated by the QA system in response to an input question to generate a ranked listing of candidate answers; inputting, by the data processing system, a training question to the QA system to generate a training candidate answer set; applying, by the data processing system, the ranker array to the training candidate answer set to generate, for each ranker in the ranker array, a ranked listing of candidate answers in the training candidate answer set; determining, by the data processing system, for each ranker in the ranker array, correctness of a corresponding ranked listing of candidate answers; and reconfiguring, by the data processing system, the ranker array based on results of the determined correctness of the corresponding ranked listing of candidate answers for each ranker in the ranker array.