Under review as a conference paper at ICLR 2018

arXiv:1801.08459v2 [cs.AI] 26 Jan 2018

F INDING R E MO (R ELATED M EMORY O BJECT ):
A S IMPLE N EURAL A RCHITECTURE
FOR T EXT BASED R EASONING
Jihyung Moon∗, Hyochang Yang, Sungzoon Cho
Department of Industrial Engineering
Seoul National University
Seoul, South Korea
{jhmoon, hyochang}@dm.snu.ac.kr, zoon@snu.ac.kr

A BSTRACT
To solve the text-based question and answering task that requires relational reasoning, it is necessary to memorize a large amount of information and find out the
question relevant information from the memory. Most approaches were based on
external memory and four components proposed by Memory Network. The distinctive component among them was the way of finding the necessary information
and it contributes to the performance. Recently, a simple but powerful neural network module for reasoning called Relation Network (RN) has been introduced.
We analyzed RN from the view of Memory Network, and realized that its MLP
component is able to reveal the complicate relation between question and object
pair. Motivated from it, we introduce Relation Memory Network (RMN) which
uses MLP to find out relevant information on Memory Network architecture. It
shows new state-of-the-art results in jointly trained bAbI-10k story-based question answering tasks and bAbI dialog-based question answering tasks.

1

I NTRODUCTION

Neural network has made an enormous progress on the two major challenges in artificial intelligence:
seeing and reading. In both areas, embedding methods have served as the main vehicle to process and
analyze text and image data for solving classification problems. As for the task of logical reasoning,
however, more complex and careful handling of features is called for. A reasoning task requires
the machine to answer a simple question upon the delivery of a series of sequential information.
For example, imagine that the machine is given the following three sentences: “Mary got the milk
there.”, “John moved to the bedroom.”, and “Mary traveled to the hallway.” Once prompted with the
question, “Where is the milk?”, the machine then needs to sequentially focus on the two supporting
sentences, “Mary got the milk there.” and “Mary traveled to the hallway.” in order to successfully
determine that the milk is located in the hallway.
Inspired by this reasoning mechanism, J. Weston & Bordes (2015) has introduced the memory network (MemNN), which consists of an external memory and four components: input feature map
(I), generalization (G), output feature map (O), and response (R). The external memory enables
the model to deal with a knowledge base without loss of information. Input feature map embeds the
incoming sentences. Generalization updates old memories given the new input and output feature
map finds relevant information from the memory. Finally, response produces the final output.
Based on the memory network architecture, neural network based models like end-to-end
memory network (MemN2N) (Sukhbaatar et al., 2015), gated end-to-end memory network
(GMemN2N) (Liu & Perez, 2017), dynamic memory network (DMN) (Kumar et al., 2016), and dynamic memory network + (DMN+) (Xiong et al., 2016) are proposed. Since strong reasoning ability
depends on whether the model is able to sequentially catching the right supporting sentences that
lead to the answer, the most important thing that discriminates those models is the way of constructing the output feature map. As the output feature map becomes more complex, it is able to
∗

Code is publicly available at: https://github.com/juung/RMN

1

Under review as a conference paper at ICLR 2018

learn patterns for more complicate relations. For example, MemN2N, which has the lowest performance among the four models, measures the relatedness between question and sentence by the
inner product, while the best performing DMN+ uses inner product and absolute difference with two
embedding matrices.
Recently, a new architecture called Relation Network (RN) (Santoro et al., 2017) has been proposed
as a general solution to relational reasoning. The design philosophy behind it is to directly capture
the supporting relation between the sentences through the multi-layer perceptron (MLP). Despite its
simplicity, RN achieves better performance than previous models without any catastrophic failure.
The interesting thing we found is that RN can also be interpreted in terms of MemNN. It is composed
of O and R where each corresponds to MLP which focuses on the related pair and another MLP
which infers the answer. RN does not need to have G because it directly finds all the supporting
sentences at once. In this point of view, the significant component would be MLP-based output
feature map. As MLP is enough to recognize highly non-linear pattern, RN could find the proper
relation better than previous models to answer the given question.
However, as RN considers a pair at a time unlike MemNN, the number of relations that RN learns
is n2 when the number of input sentence is n. When n is small, the cost of learning relation is
reduced by n times compared to MemNN based models, which enables more data-efficient learning (Santoro et al., 2017). However, when n increases, the performance becomes worse than the
previous models. In this case, the pair-wise operation increases the number of non-related sentence
pairs more than the related sentence pair, thereby confuses RN’s learning. Santoro et al. (2017) has
suggested attention mechanisms as a solution to filter out unimportant relations; however, since it
interrupts the reasoning operation, it may not be the most optimal solution to the problem.
Our proposed model, “Relation Memory Network” (RMN), is able to find complex relation even
when a lot of information is given. It uses MLP to find out relevant information with a new generalization which simply erase the information already used. In other words, RMN inherits RN’s
MLP-based output feature map on Memory Network architecture. Experiments show its state-ofthe-art result on the text-based question answering tasks.

2

R ELATION M EMORY N ETWORK

ൈ

$%&'()*+

klm



YZ

18 Mary travelled to the kitchen.

σ

efgh

145 Mary went to the bedroom.

QRST

abcd

146 Daniel travelled to the hallway.

MNOP

]^_`

147 Daniel went back to the garden.

KL

[\

148 Where was the apple before

the garden?

͙

͙

͙

͙

͙

͙

UVWX

144 Daniel grabbed the apple.

56789:;<=
>?@ABC

ij

σ

hallway

,-./01234
D

 !"#


FG

E

HI

J

Figure 1: Relation Memory Network

Relation Memory Network (RMN) is composed of four components - embedding, attention, updating, and reasoning. It takes as the inputs a set of sentences x1 , x2 , ..., xn and its related question u,
and outputs an answer a. Each of the xi , u, and a is made up of one-hot representation of words,
for example, xi = {xi1 , xi2 , xi3 , ..., xini } (xij ∈ RV , j = (1, 2, ..., ni ), V = vocabulary size, ni =
number of words in sentence i).
2

Under review as a conference paper at ICLR 2018

2.1

E MBEDDING

COMPONENT

We first embed words in each xi = {xi1 , xi2 , xi3 , ..., xini } and u to a continuous space multiplying
an embedding matrix A ∈ Rd×V . Then, the embedded sentence is stored and represented as a
memory object mi while question is represented as q. Any of the following methods are available
for embedding component: simple sum (equation 1), position encoding (J. Weston & Bordes, 2015)
(equation 2), concatenation (equation 3), LSTM, and GRU. In case of LSTM or GRU, mi is the
final hidden state of it.
X
Axij
(1)
mi =
j

mi =

X

lj · Axij

(lkj = (1 − j/ni ) − (k/d)(1 − 2j/ni ))

(2)

j

mi = [Axi1 , Axi2 , ... , Axini ]

(3)

As the following attention component takes the concatenation of mi and q, it is not necessarily
the case that sentence and question have the same dimensional embedding vectors unlike previous
memory-augmented neural networks.
2.2

ATTENTION COMPONENT

Attention component can be applied more than once depending on the problem; Figure 1 illustrates
2 hop version of RMN. We refer to the ith embedded sentence on the tth hop as mti .
To constitute the attention component, we applied simple MLP represented as gθt . It must be ended
with 1 unit output layer to provide a scalar weight wit , which leads to an attention weight αti between
0 and 1. In the beginning, a vector concatenated with m1i and q flows to the gθ1 . From the result of
gθ1 , attention weight α1i is calculated using additional variable β 1 (≥ 1) to control the intensity of
attention, inspired by the way Neural Turing Machine (Graves et al., 2014) reads from the memory.
Then we get the related memory object r1 , a weighted sum of α1i and memory object m1i for all i.
If there exist more hops, r1 is directly taken to the next hop and iterates over this process with the
updated memory object m2i . All the procedures are rewritten as equation 4, 5, and 6:
wit ← gθt ([mti , rt−1 ])
exp(β t wit )
αti ← P
t t
i exp(β wi )

(i = (1, 2, ..., n), r0 = q)

(4)

(β t (z) = 1 + log(1 + exp(z)))

(5)

rt ←

X

αti · mti

(6)

i

2.3

U PDATING

COMPONENT

To forget the information already used, we use intuitive updating component to renew the memory.
It is replaced by the amount of unconsumed from the old one:
mt+1
← (1 − αti ) mti
i

(7)

Contrary to other components, updating is not a mandatory component. When it is considered to
have 1 hop, there is no need to use this.
2.4

R EASONING

COMPONENT

Similar to attention component, reasoning component is also made up of MLP, represented as fφ . It
receives both q and the final result of attention component rf and then takes a softmax to produce
the model answer â:
â ← Softmax(fφ ([rf , q]))
(8)
3

Under review as a conference paper at ICLR 2018

Table 1: MemN2N, RN, and RMN in terms of MemNN architecture
Output feature map
MemN2N

αti

=

t T

t
0
Softmax((r
P ) tmit)(r
t
o = i αi mi
t+1
t
t

r

3
3.1

= q)

mt = W t mt−1

=o +r

ri = gθ -MLP([m
P i , mj , q])
o = i ri

-

rti = gθt -MLP([mti , rt−1 ])(r0 = q)
t t
αti = Softmax(β
P t t ri )
t
r = i αi mi

mti = (1 − αt−1
)mt−1
i
i

RN

RMN

Generalization

R ELATED WORK
M EMORY-AUGMENTED N EURAL N ETWORK

To answer the question from a given set of facts, the model needs to memorize these facts from the
past. Long short term memory (LSTM) (Hochreiter & Schmidhuber, 1997), one of the variants of
recurrent neural network (RNN), is inept at remembering past stories because of their small internal memory (Sukhbaatar et al., 2015). To cope with this problem, J. Weston & Bordes (2015) has
proposed a new class of memory-augmented model called Memory Network (MemNN). MemNN
comprises an external memory m and four components: input feature map (I), generalization (G),
output feature map (O), and response (R). I encodes the sentences which are stored in memory m.
G updates the memory, whereas O reads output feature o from the memory. Finally, R infers an
answer from o.
MemN2N, GMemN2N, DMN, and DMN+ all follow the same structure of MemNN from a broad
perspective, however, output feature map is composed in slightly different way. The relation between
question and supporting sentences is realized from its cooperation. MemN2N first calculates the
relatedness of sentences in the question and memory by taking the inner product, and the sentence
with the highest relatedness is selected as the first supporting sentence for the given question. The
first supporting sentence is then added with the question and repeat the same operation with the
updated memory to find the second supporting sentence. GMemN2N selects the supporting sentence
in the same way as MemN2N, but uses the gate to selectively add the the question to control the
influence of the question information in finding the supporting sentence in the next step. DMN and
DMN + use output feature map based on various relatedness such as absolute difference, as well as
inner product, to understand the relation between sentence and question at various points.
The more difficult the task, the more complex the output feature map and the generalization component to get the correct answer. For a dataset experimenting the text-based reasoning ability of
the model, the overall accuracy could be increased in order of MemN2N, GMemN2N, DMN, and
DMN+, where the complexity of the component increases.
3.2

R ELATION N ETWORK

Relation Network (RN) has emerged as a new and simpler framework for solving the general reasoning problem. RN takes in a pair of objects as its input and simply learns from the compositions
of two MLPs represented as gθ and fφ . The role of each MLP is not clearly defined in the original
paper, but from the view of MemNN, it can be understood that gθ corresponds to O and fφ corresponds to R. Table 1 summarizes the interpretation of RN compared to MemN2N and our model,
RMN.
To verify the role of gθ , we compare the output when pairs are made with supporting sentences
and when made with unrelated sentences. Figure 2 shows the visualization result of each output.
When we focus on whether the value is activated or not, we can see that gθ distinguishes supporting
sentence pair from non-supporting sentence pair as output feature map examines how relevant the
4

Under review as a conference paper at ICLR 2018

sentence is to the question. Therefore, we can comprehend the output of gθ reveals the relation
between the object pair and the question and fφ aggregates all these outputs to infer the answer.

(a) related objects

(b) unrelated objects

Figure 2: The output vector of gθ when the input objects are related and unrelated

4
4.1

E XPERIMENTS
DATASET

bAbI story-based QA dataset bAbI story-based QA dataset (Weston et al., 2015) is composed
of 20 different types of tasks for testing natural language reasoning ability. Each task requires different methods to infer the answer. The dataset includes a set of statements comprised of multiple
sentences, a question and answer. A statement can be as short as two sentences and as long as 320
sentences. To answer the question, it is necessary to find relevant one or more sentences to a given
question and derive answer from them. Answer is typically a single word but in a few tasks, answers
are a set of words. Each task is regarded as success when the accuracy is greater than 95%. There
are two versions of this dataset, one that has 1k training examples and the other with 10k examples.
Most of the previous models test their accuracy on 10k dataset with trained jointly.
bAbI dialog dataset bAbI dialog dataset (Bordes & Weston, 2016) is a set of 5 tasks within
the goal-oriented context of restaurant reservation. It is designed to test if model can learn various
abilities such as performing dialog management, querying knowledge bases (KBs), and interpreting
the output of such queries. The KB can be queried using API calls and 4 fields (a type of cuisine,
a location, a price range, and a party size). They should be filled to issue an API call. Task 1 tests
the capacity of interpreting a request and asking the right questions to issue an API call. Task 2
checks the ability to modify an API call. Task 3 and 4 test the capacity of using outputs from an API
call to propose options in the order of rating and to provide extra-information of what user asks for.
Task 5 combines everything. The maximum length of the dialog for each task is different: 14 for
task 1, 20 for task 2, 78 for task 3, 13 for task 4, and 96 for task 5. As restaurant name, locations,
and cuisine types always face new entities, there are normal and OOV test sets to assess model’s
generalization ability. Training sets consist fo 1k examples, which is not a large amount of creating
realistic learning conditions.
4.2

T RAINING

DETAILS

bAbI story-based QA dataset We trained 2 hop RMN jointly on all tasks using 10k dataset for
model to infer the solution suited to each type of tasks. We limited the input to the last 70 stories
for all tasks except task 3 for which we limited input to the last 130 stories, similar to Xiong et al.
(2016) which is the hardest condition among previous models. Then, we labeled each sentence with
its relative position. Embedding component is similar to Santoro et al. (2017), where story and question are embedded through different LSTMs; 32 unit word-lookup embeddings; 32 unit LSTM for
story and question. For attention component, as we use 2 hop RMN, there are gθ1 and gθ2 ; both
are three-layer MLP consisting of 256, 128, 1 unit with ReLU activation function (Nair & Hinton,
2010). fφ is composed of 512, 512, and 159 units (the number of words appearing in bAbI dataset
is 159) of three-layer MLP with ReLU non-linearities where the final layer was a linear that produced logits for a softmax over the answer vocabulary. For regularization, we use batch normalization (Ioffe & Szegedy, 2015) for all MLPs. The softmax output was optimized with a cross-entropy
loss function using the Adam optimizer (Kingma & Ba, 2014) with a learning rate of 2e−4 .
bAbI dialog dataset We trained on full dialog scripts with every model response as answer, all
previous dialog history as sentences to be memorized, and the last user utterance as question. Model
selects the most probable response from 4,212 candidates which are ranked from a set of all bot
utterances appearing in training, validation and test sets (plain and OOV) for all tasks combined.
We also report results when we use match type features for dialog. Match type feature is an additional label on the candidates indicating if word is found on the dialog history. For example, if the
5

Under review as a conference paper at ICLR 2018

Table 2: Test error on bAbI story-based tasks with 10k training samples
Task
1: Single Supporting Fact
2: Two Supporting Facts
3: Three Supporting Facts
4: Two Argument Relations
5: Three Argument Relations
6: Yes/No Questions
7: Counting
8: Lists/Sets
9: Simple Negation
10: Indefinite Knowledge
11: Basic Coreference
12: Conjunction
13: Compound Coreference
14: Time Reasoning
15: Basic Deduction
16: Basic Induction
17: Positional Reasoning
18: Size Reasoning
19: Path Finding
20: Agent’s Motivations
Mean error (%)
Failed tasks (err. >5%)

MemNN
0.0
0.0
0.0
0.0
2.0
0.0
15.0
9.0
0.0
2.0
0.0
0.0
0.0
1.0
0.0
0.0
35.0
5.0
64.0
0.0
6.7
4

MemN2N
0.0
0.3
9.3
0.0
0.6
0.0
3.7
0.8
0.8
2.4
0.0
0.0
0.0
0.0
0.0
0.4
40.7
6.7
66.5
0.0
6.6
4

GMemN2N
0.0
0.0
4.5
0.0
0.2
0.0
1.8
0.3
0.0
0.2
0.0
0.0
0.0
0.0
0.0
0.0
27.8
8.5
31.0
0.0
3.7
3

DMN
0.0
1.8
4.8
0.0
0.7
0.0
3.1
3.5
0.0
2.5
0.1
0.0
0.2
0.0
0.0
0.6
40.4
4.7
65.5
0.0
6.4
2

DMN+
0.0
0.3
1.1
0.0
0.5
0.0
2.4
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
45.3
4.2
2.1
0.0
0.0
2.8
1

DNC
0.0
0.4
1.8
0.0
0.8
0.0
0.6
0.3
0.2
0.2
0.0
0.0
0.1
0.4
0.0
33.1
12.0
0.8
3.9
0.0
2.7
2

EntNet1
0.1
2.8
10.6
0.0
0.4
0.3
0.8
0.1
0.0
0.0
0.0
0.0
0.0
3.6
0.0
52.1
11.7
2.1
63.0
0.0
7.4
4

RN2
0.0
8.3
17.1
0.0
0.7
0.0
0.4
0.3
0.0
0.0
0.4
0.0
0.0
0.0
0.0
4.9
1.6
2.1
3.2
0.0
2.0
2

RMN
0.0
0.5
14.7
0.0
0.4
0.0
0.5
0.3
0.0
0.0
0.5
0.0
0.0
0.0
0.0
0.9
0.3
2.3
2.9
0.0
1.2
1

world ‘Seoul’ is found, then the ‘location’ field is checked to hint model this word is important and
should be used in API call. This feature can alleviate OOV problem. Training was done with Adam
optimizer and a learning rate of 1e−4 for all tasks. Additional model details are given in Appendix A.

5

R ESULTS

5.1

AND

D ISCUSSION

BA B I STORY- BASED

QA

As we can see in table 2, RMN shows state-of-the-art result on bAbI story-based Question Answering dataset: 98.8% accuracy where only a single task with no catastrophic failure. It succeeded on
task 17, 18, 19 where MemN2N, GMemN2N, and DMN are failed to solve, and on task 16 which
DMN+, DNC, and EntNet scored high error rates.
Table 3 shows how our model solved several tasks. RMN’s attention component gθ1 and gθ2 complement each other to identify the necessary facts to answer correctly. Sometimes both gθ1 and gθ2
concentrate on the same sentences which are all critical to answer the question, and sometimes gθ1
finds a fact related to the given question and with this information gθ2 chooses the key fact to answer. While trained jointly, RMN learns these different solutions for each task. For the task 3, the
only failed task, attention component still functions well; it focuses sequentially on the supporting
sentences. However, the reasoning component, fφ , had difficulty catching the word ‘before’. We
could easily figure out ‘before’ implies ‘just before’ the certain situation, whereas RMN confused
its meaning. As shown in table 3c, our model found all previous locations before the garden. Still, it
is remarkable that the simple MLP carried out all of these various roles.
5.2

BA B I DIALOG

The results in the Table 4 show that the RMN has the best results in any conditions. Without any
match type, RN and RMN outperform previous memory-augmented models on both normal and
OOV tasks. This is mainly attributed to the impressive result on task 4 which can be interpreted as
an effect of MLP based output feature map.
To solve task 4, it is critical to understand the relation between ‘phone number’ of user input and
‘r phone’ of previous dialog as shown in Table 8c. We assumed that inner product was not sufficient
1

For a fair comparison, we report EntNet’s result which was jointly trained on all tasks. It was written in
the appendix of the paper.
2
Our implementation. The result is different from what Santoro et al. (2017) mentioned, which is caused
by the initialization.

6

Under review as a conference paper at ICLR 2018

Table 3: bAbI story-based task visualization of α
(c) Task 3

(a) Task 7
Seq.
8
9
10
11
13
14
15
16
User input
Answer
Model answer

Task 7: Counting
John grabbed the apple there .
John gave the apple to Mary .
Mary passed the apple to John .
Mary journeyed to the hallway .
Sandra went to the garden .
Mary went to the kitchen .
Mary picked up the football there .
Mary picked up the milk there .
How many objects is Mary carrying?
Two
Two

α1
0.02
0.08
0.17
0.00
0.00
0.00
0.29
0.27

α2
0.00
0.16
0.24
0.01
0.00
0.03
0.24
0.32

Seq.
33
39
40
41
42
46
50
51
User input
Answer
Model answer
Seq.
1
3
5
6
11
12
14
16
22
User input
Answer
Model answer

[Correct]

(b) Task 14
Seq.
1
2
3
4
5
6
7
User input
Answer
Model answer

Task 14: Time reasoning
α1
Mary went back to the school yesterday .
0.00
Fred went to the school yesterday .
0.00
Julie went back to the kitchen yesterday .
0.13
Fred journeyed to the kitchen this morning .
0.00
This morning julie journeyed to the school .
0.66
This evening mary went back to the school .
0.01
This afternoon julie went to the bedroom .
0.06
Where was Julie before the school ?
Kitchen
Kitchen
[Correct]

α2
0.01
0.00
0.98
0.00
0.02
0.00
0.00

Task 3: Three Supporting Facts
α1
Daniel took the football .
0.46
Sandra travelled to the bedroom .
0.01
Daniel moved to the bathroom .
0.01
Sandra got the milk .
0.00
Daniel travelled to the garden .
0.04
Daniel went to the hallway .
0.00
Daniel put down the apple .
0.00
Daniel put down the football there .
0.38
Where was the football before the garden ?
Bathroom
Bathroom
[Correct]
Task 3: Three Supporting Facts
α1
Mary got the football .
0.26
Mary picked up the football .
0.39
Mary moved to the office .
0.00
Mary went to the hallway .
0.00
Mary travelled to the garden .
0.00
Mary travelled to the kitchen .
0.00
Mary moved to the office .
0.00
Mary went to the garden .
0.00
Mary discarded the football there .
0.24
Where was the football before the garden ?
Office
Kitchen
[Incorrect]

α2
0.01
0.00
0.31
0.00
0.42
0.25
0.01
0.00

α2
0.00
0.00
0.05
0.05
0.21
0.29
0.25
0.05
0.00

Table 4: Test error on bAbI dialog tasks 3
Task
1: Issuing API calls
2: Updating API calls
3: Displaying options
4: Providing extra information
5: Conducting full dialogs
Average error rates (%)
1 (OOV): Issuing API calls
2 (OOV): Updatining API calls
3 (OOV): Displaying options
4 (OOV): Providing extra information
5 (OOV): Conducting full dialogs
Average error rates (%)

MemN2N
0.1
0.0
25.1
40.5
3.9
13.9
27.7
21.1
25.6
42.4
34.5
30.3

Plain
GMemN2N
0.0
0.0
25.1
42.8
3.7
14.3
17.6
21.1
24.7
43.0
33.3
27.9

RN4
0.0
0.5
26.6
0.0
23.3
10.1
17.8
23.2
27.2
0.0
38.3
21.3

RMN
0.0
0.0
25.1
0.0
2.5
5.5
16.8
21.1
24.9
0.0
34.5
19.5

MemN2N
0.0
1.7
25.1
0.0
6.6
6.7
3.5
5.5
24.8
0.0
22.3
11.2

With Match
GMemN2N
0.0
0.0
25.1
0.0
2.0
5.4
0.0
5.8
24.9
0.0
20.6
10.3

RN4
0.0
0.0
27.1
0.0
16.6
8.7
1.5
0.0
29.8
0.0
28.4
12.0

RMN
0.0
0.0
25.1
0.0
1.8
5.4
0.0
0.0
25.1
0.0
21.7
9.4

to capture their implicit similarity and performed an supporting experiment. We converted RMN’s
attention component to inner product based attention, and the results revealed the error rate increased
to 11.3%.
For the task 3 and task 5 where the maximum length is especially longer than the others, RN performs worse than MemN2N, GMemN2N and RMN. The number of unnecessary object pairs created
by the RN not only increases the processing time but also decreases the accuracy.
With the match type feature, all models other than RMN have significantly improved their performance except for task 3 compared to the plain condition. RMN was helped by the match type only
on the OOV tasks and this implies RMN is able to find relation in the With Match condition for the
normal tasks.
When we look at the OOV tasks more precisely, RMN failed to perform well on the OOV task 1
and 2 even though gθ1 properly focused on the related object as shown in Table 8a. We state that this
originated from the fact that the number of keywords in task 1 and 2 is bigger than that in task 4. In
task 1 and 2, all four keywords (cuisine, location, number and price) must be correctly aligned from
the supporting sentence in order to make the correct API call which is harder than task 4. Consider
the example in Table 8a and Table 8c. Supporting sentence of task 4 have one keyword out of three
words, whereas supporting sentences of task 1 and 2 consist of four keywords (cuisine, location,
number and price) out of sixteen words.
3
4

We only compare models under the same conditions
Our implementation.

7

Under review as a conference paper at ICLR 2018

Different from other tasks, RMN yields the same error rate 25.1% with MemN2N and GMemN2N
on the task 3. The main goal of task 3 is to recommend restaurant from knowledge base in the
order of rating. All failed cases are displaying restaurant where the user input is <silence>which is
somewhat an ambiguous trigger to find the input relevant previous utterance. As shown in Table 8b,
there are two different types of response to the same user input. One is to check whether all the
required fields are given from the previous utterances and then ask user for the missing fields or
send a “Ok let me look into some options for you.” message. The other type is to recommend
restaurant starting from the highest rating. All models show lack of ability to discriminate these two
types of silences so that concluded to the same results. To verify our statement, we performed an
additional experiment on task 3 and checked the performance gain (extra result is given in Table 10
of Appendix B).
5.3

M ODEL ANALYSIS

Effectiveness of the MLP-based output feature map The most important feature that distinguishes MemNN based models is the output feature map. Table 5 summarizes the experimental
results for the bAbI story-based QA dataset when replacing the RMN’s MLP-based output feature
map with the idea of the previous models. inner product was used in MemN2N, inner product with
gate was used in GMemN2N, and inner product and absolute difference with two embedding matrices was used in DMN and DMN+. From the Table 5, the more complex the output feature map, the
better the overall performance. In this point of view, MLP is the effective output feature map.
Table 5: Test error of RMN on bAbI story-based QA dataset with different configurations

error rate

inner product

inner product with gate

29.4

25.9

inner product and absolute difference
with two embedding matrices
11.2

MLP
1.2

Performance of RN and RMN according to memory size Additional experiments were conducted with the bAbI story-based QA dataset to see how memory size affects both performance and
training time of RN and RMN. Test errors with training time written in parentheses are summarized
in Table 6.
When memory size is small, we could observe the data-effeciency of RN. It shows similar performance to RMN in less time. However, when the memory size increases, performance is significantly
reduced compared to RMN, even though it has been learned for a longer time. It is even lower than
itself when the memory size is 20. On the other hand, RMN maintains high performance even when
the memory size increases.
Table 6: Test error and training time of RN and RMN on bAbI story-based QA dataset with different
memory size
memory size
20
130

RN
2.0 (0.65 days)
9.8 (9.47 days)

RMN
1.5 (1.46 days)
1.2 (4.94 days)

Effectiveness of the number of hops bAbI story based QA dataset differs in the number of supporting sentences by each task that need to be referenced to solve problems. For example, task 1, 2,
and 3 require single, two, and three supporting facts, respectively. The result of the mean error rate
for each task according to the number of hops is in Table 7.
Table 7: Test error of RMN on bAbI story-based QA dataset with different number of hops
task 1 (1) 5
task 2 (2)
task 3 (3)
task 10 (1)

hop 1
0.0
62.0
62.4
0.0
8

hop 2
0.0
0.5
14.7
0.0

hop 3
0.2
2.1
14.6
3.6

Under review as a conference paper at ICLR 2018

Overall, the number of hops is correlated with the number of supporting sentences. In this respect,
when the number of relations increases, RMN could reason across increasing the number of hops to
3, 4 or more.

6

C ONCLUSION

Our work, RMN, is a simple and powerful architecture that effectively handles text-based question
answering tasks when large size of memory and high reasoning ability is required. Multiple access
to the external memory to find out necessary information through a multi-hop approach is similar to
most existing approaches. However, by using a MLP that can effectively deal with complex relatedness when searching for the right supporting sentences among a lot of sentences, RMN raised the
state-of-the-art performance on the story-based QA and goal-oriented dialog dataset. When comparing RN which also used MLP to understand relations, RMN was more effective in the case of large
memory.
Future work will apply RMN to image based reasoning task (e.g., CLEVR, DAQUAR, VQA etc.).
To extract features from the image, VGG net (Simonyan & Zisserman, 2014) is used in convention
and outputs 196 objects of 512 dimensional vectors which also require large sized memory. An
important direction will be to find an appropriate way to focus sequentially on related object which
was rather easy in text-based reasoning.

5

Number in parentheses indicates the number of supporting sentences to solve the task

9

Under review as a conference paper at ICLR 2018

Table 8: Visualization of α on bAbI dialog task 1, 3, and 4 without match type
(a) Plain and OOV result of task 1
Seq. Locutor
1
user
2
bot
3
user
4
bot
5
user
6
bot
7
user
8
bot
User input
Answer
Model answer
Seq. Locutor
1
user
2
bot
3
user
4
bot
5
user
6
bot
User input
Answer
Model answer

Task 1: Issuing API calls
Good morning
Hello what can i help you with today
Can you make restaurant reservation for eight people in cheap price range with british cuisine
I’m on it
<silence>
Where should it be
In madrid
Ok let me look into some options for you
<silence>
API call british madrid eight cheap
API call british madrid eight cheap
[Correct]
Task 1 (OOV): Issuing API calls
Hi
Hello what can i help you with today
May I have table with cantonese cuisine in cheap price range in seoul for two people
I’m on it
<silence>
Ok let me look into some options for you
<silence>
API call cantonese seoul two cheap
API call italian paris two cheap
[Incorrect]

Seq. Locutor
7
user
14
user
21
user
22
user
23
bot
24
user
25
bot
User input
Answer
Model answer
Seq. Locutor
7
user
14
user
21
user
24
user
25
bot
26
user
27
bot
28
user
29
bot
30
user
31
bot
User input
Answer
Model answer

Task 3: Displaying options
resto 1 r rating 1
resto 8 r rating 8
resto 3 r rating 3
Hello
Hello what can i help you with today
Can you book table with french food for two people in madrid in moderate price range
I’m on it
<silence>
Where should it be?
Where should it be?
[Correct]
Task 3: Displaying options
resto 1 r rating 1
resto 8 r rating 8
resto 3 r rating 3
Can you book table with french food for two people in madrid in moderate price range
I’m on it
<silence>
Ok let me look into some options for you
<silence>
What do you think of this option : resto 8
No this does not work for me
Sure let me find other option for you
<silence>
What do you think of this option: resto 3
What do you think of this option: resto 7
[Incorrect]

α1
0.01
0.01
0.23
0.00
0.14
0.01
0.25
0.18

α1
0.01
0.01
0.49
0.10
0.01
0.27

(b) Same user input with different goal in task 3
α1
0.01
0.01
0.01
0.01
0.01
0.21
0.01

α1
0.00
0.00
0.00
0.14
0.00
0.09
0.15
0.10
0.08
0.06
0.08

(c) Plain and OOV result of task 4
Seq. Locutor
1
user
3
user
4
user
6
user
10
user
11
bot
User input
Answer
Model answer
Seq. Locutor
1
user
3
user
4
user
6
user
10
user
11
bot
User input
Answer
Model answer

Task 4: Providing extra information
resto 3 r phone resto 3 phone
resto 3 r address resto 3 address
resto 3 r location london
resto 3 r price cheap
I’d like to book table at resto 3
Great let me do reservation
What is phone number of restaurant?
Here it is: resto 3 phone
Here it is: resto 3 phone
Task 4 (OOV): Providing extra information
resto 1 r phone resto 1 phone
resto 1 r address resto 1 address
resto 1 r location hanoi
resto 1 r price expensive
Can you make restaurant reservation at resto 1
Great let me do reservation
What is phone number of restaurant?
Here it is: resto 1 phone
Here it is: resto 1 phone

10

α1
0.77
0.01
0.01
0.01
0.02
0.03

[Correct]
α1
0.79
0.01
0.01
0.01
0.01
0.04

[Correct]

Under review as a conference paper at ICLR 2018

R EFERENCES
Antoine Bordes and Jason Weston.
abs/1605.07683, 2016.

Learning end-to-end goal-oriented dialog.

Alex Graves, Greg Wayne, and Ivo Danihelka.
arXiv:1410.5401, 2014.

Neural turing machines.

CoRR,

arXiv preprint

Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):
1735–1780, 1997.
Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by
reducing internal covariate shift. In International Conference on Machine Learning, pp. 448–
456, 2015.
S. Chopra J. Weston and A. Bordes. Memory networks. International Conference on Learning
Representations (ICLR), 2015.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint
arXiv:1412.6980, 2014.
Ankit Kumar, Ozan Irsoy, Peter Ondruska, Mohit Iyyer, James Bradbury, Ishaan Gulrajani, Victor
Zhong, Romain Paulus, and Richard Socher. Ask me anything: Dynamic memory networks for
natural language processing. In International Conference on Machine Learning, pp. 1378–1387,
2016.
Fei Liu and Julien Perez. Gated end-to-end memory networks. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long
Papers, volume 1, pp. 1–10, 2017.
Vinod Nair and Geoffrey E Hinton. Rectified linear units improve restricted boltzmann machines. In
Proceedings of the 27th international conference on machine learning (ICML-10), pp. 807–814,
2010.
Adam Santoro, David Raposo, David GT Barrett, Mateusz Malinowski, Razvan Pascanu, Peter
Battaglia, and Timothy Lillicrap. A simple neural network module for relational reasoning. Advances in neural information processing systems, 2017.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image
recognition. arXiv preprint arXiv:1409.1556, 2014.
Sainbayar Sukhbaatar, Jason Weston, Rob Fergus, et al. End-to-end memory networks. In Advances
in neural information processing systems, pp. 2440–2448, 2015.
Jason Weston, Antoine Bordes, Sumit Chopra, and Tomas Mikolov. Towards ai-complete question
answering: A set of prerequisite toy tasks. CoRR, abs/1502.05698, 2015.
Caiming Xiong, Stephen Merity, and Richard Socher. Dynamic memory networks for visual and
textual question answering. In International Conference on Machine Learning, pp. 2397–2406,
2016.

11

Under review as a conference paper at ICLR 2018

A

M ODEL

DETAILS

Table 9: Hyperparameters of Relation Memory Networks on bAbI dialog tasks
Task
1
2
3
4
5

B

Story and Question
Embedding
sum
sum
sum
concatenation
concatenation

Word-lookup
Embedding Dim
128
128
128
50
64

Hop

gθ

fφ

Activation

Use Batch Norm

1
1
1
1
2

2048, 2048, 1
1024, 1024, 1
1024, 1024, 1024, 1
1024, 1024, 1
4096, 4096, 1

2048, 2048, 4212
1024, 1024, 4212
1024, 1024, 1024, 4212
1024, 1024, 4212
4096, 4096, 4212

tanh
tanh
tanh
tanh
tanh

True
True
True
True
True

A DDITIONAL R ESULTS

Table 10: Visualization of α1 and α2 on user input revised bAbI dialog task 3 without match type
Seq. Locutor
1
user
2
user
3
user
4
user
5
user
6
user
7
user
8
user
9
user
10
user
11
user
12
user
13
user
14
user
15
user
16
user
17
user
18
user
19
user
20
user
21
user
22
user
23
bot
24
user
25
bot
26
user
27
bot
28
user
29
bot
30
user
31
bot
User input
Answer
Model answer

Task 3: Displaying options
resto 8 r phone resto 8 phone
resto 8 r cuisine french
resto 8 r address resto 8 address
resto 8 r location madrid
resto 8 r number two
resto 8 r price moderate
resto 8 r rating 8
resto 3 r phone resto 3 phone
resto 3 r cuisine french
resto 3 r address resto 3 address
resto 3 r location madrid
resto 3 r number two
resto 3 r price moderate
resto 3 r rating 3
resto 1 r phone resto 1 phone
resto 1 r cuisine french
resto 1 r address resto 1 address
resto 1 r location madrid
resto 1 r number two
resto 1 r price moderate
resto 1 r rating 1
Good morning
Hello what can I help you with today?
May I have table for two with french food?
I’m on it
<silence>
Where should it be
In madrid
Which price range are looking for?
In moderate price range please
Ok let me look into some options for you
<silence><silence>
What do you think of this option: resto 8
What do you think of this option: resto 8

α1
0.00
0.01
0.01
0.01
0.00
0.00
0.01
0.01
0.00
0.00
0.00
0.00
0.00
0.00
0.01
0.01
0.01
0.01
0.00
0.00
0.00
0.01
0.12
0.08
0.03
0.01
0.09
0.01
0.09
0.01
0.24

α2
0.01
0.01
0.01
0.01
0.01
0.01
0.39
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.02
0.01
0.01
0.01
0.01
0.01
0.01
0.01

[correct]

We modify the user input from <silence>to <silence><silence>when looking for restaurant recommendations. This makes model to distinguish two different situations whether to ask for additional fields or to recommend restaurant.

12

