1. A method, in a question and answer (QA) system comprising a processor and a memory, for retrieving candidate answers from a corpus of documents, the method comprising: receiving, by the QA system, an input question for which an answer is sought; extracting, by the QA system, features of the input question based on a natural language processing of the input question; executing, by the QA system, a first search of the corpus of documents based on a first subset of the extracted features of the input question and an initial evaluation of a utility of the first subset of extracted features to generate a subset of documents matching the first subset of extracted features, wherein the utility of the first subset of extracted features identifies a degree to which each feature of the first subset of extracted features of the input question discriminates between documents in the corpus of documents that are sources of candidate answers to the input question; executing, by the QA system, a second search of a set of passages extracted from the subset of documents based on a second subset of the extracted features of the input question and a reevaluation of the utility of the second subset of extracted features thereby forming a subset of passages, wherein the utility of the second subset of extracted features identifies a degree to which each feature of the second subset of extracted features of the input question discriminates between passages in the set of passages that are sources of candidate answers to the input question; and generating, by the QA system, query results from the subset of passages from which a set of candidate answers for the input question are identified.