1. A method, in a data processing system comprising a processor and a memory, for modifying an operation of a question answering (QA) system, comprising: receiving, by the data processing system, an input question; processing, by the data processing system, the input question to generate at least one query to be applied to a corpus of information; applying, by the data processing system, the at least one query to the corpus of information to generate candidate answers to the input question; selecting, by the data processing system, a final answer from the candidate answers for output; and modifying, by a training engine associated with the data processing system, using a machine learning technique that compares the final answer to a known correct answer known to be a correct answer for the input question, at least one of logic or configuration parameters of the QA system for at least one of the processing of the input question to generate the at least one query, applying of the at least one query to the corpus of information to generate the candidate answers, or the selecting of the final answer from the candidate answers.