1. A method implemented by an information handling system that includes a memory and a processor, the method comprising: selecting a question submitted to a question answering (QA) system, wherein the question pertains to a domain specific corpus ingested by the QA system; analyzing data pertaining to a plurality of answers generated by the QA system, wherein the analysis results in a stability characteristic pertaining to each of the plurality of answers, and wherein the analyzing comprises: analyzing one or more characteristics of answer change pertaining to each of the plurality of answers, wherein the analysis results in a plurality of change frequencies pertaining to each of the plurality of answers; and computing, for each of the plurality of answers, a stability score related to the stability characteristic, by combining the plurality of change frequencies; adjusting a confidence value associated with each of the plurality of answers based on the respective answer's stability characteristic; and providing one or more of the plurality of answers to a requestor based on the adjusted confidence values.