1. A method, in a data processing system comprising a processor and a memory and implementing training logic for training a Question and Answer (QA) system of the data processing system, the method comprising: receiving, by the QA system, a training question; processing, by the QA system, the training question to generate a final answer to the training question and supporting evidence for the final answer based on a corpus of information; analyzing, by the training logic, the supporting evidence to identify one or more evidence attributes; searching, by the training logic, a plurality of correct answer entries in the answer key associated with the training question, to identify a matching correct answer entry that matches the final answer; augmenting the matching correct answer entry in the answer key to include the one or more evidence attributes in an augmented answer key; and training, by the training logic, the QA system based on the augmented answer key to generate a trained QA system.