We present a novel neural architecture for answering queries, designed to optimally leverage explicit support in the form of query-answer memories. Our model is able to refine and update a given query while separately accumulating evidence for predicting the answer. Its architecture reflects this separation with dedicated embedding matrices and loosely connected information pathways (modules) for updating the query and accumulating evidence. This separation of responsibilities effectively decouples the search for query related support and the prediction of the answer. On recent benchmark datasets for reading comprehension, our model achieves state-of-the-art results. A qualitative analysis demonstrates that the model effectively accumulates weighted evidence from the query and over multiple support retrieval cycles which results in a robust answer prediction.
@cite_16: 
@cite_7: Most tasks in natural language processing can be cast into question answering (QA) problems over language input. We introduce the dynamic memory network (DMN), a neural network architecture which processes input sequences and questions, forms episodic memories, and generates relevant answers. Questions trigger an iterative attention process which allows the model to condition its attention on the inputs and the result of previous iterations. These results are then reasoned over in a hierarchical recurrent sequence model to generate answers. The DMN can be trained end-to-end and obtains state-of-the-art results on several types of tasks and datasets: question answering (Facebook's bAbI dataset), text classification for sentiment analysis (Stanford Sentiment Treebank) and sequence modeling for part-of-speech tagging (WSJ-PTB). The training for these different tasks relies exclusively on trained word vector representations and input-question-answer triplets.
@cite_11: Described herein are systems and methods for providing a natural language comprehension system (NLCS) that iteratively performs an alternating search to gather information that may be used to predict the answer to the question. The NLCS first attends to a query glimpse of the question, and then finds one or more corresponding matches by attending to a text glimpse of the text.
One important contribution of Memory Networks is the idea of refining or updating the query @cite_16 or memories @cite_7 for multiple memory retrieval cycles before answering the query. This idea lead to significant improvements for architectures that employ the attention mechanism iteratively for reading comprehension tasks @cite_11 .