1. A method, in an information handling system comprising a processor and a memory, of improving a quality of question-answer sets used as inputs to a question-answering (QA) system, the method comprising: analyzing a question-answer pair using natural language processing (NLP) components, wherein the question-answer pair includes a question and an answer to the question, and wherein at least one of the NLP components is a configurable analyzer extension; identifying, based on the analysis, one or more shortcomings of the question-answer pair, wherein the shortcomings relate to an ability of the target QA system to analyze the question; and providing a human-readable feedback to a user, wherein the feedback recommends one or more possible actions to address the identified shortcomings.