title: Seq2Seq model learns to only output EOS token (<\s>) after a few iterations
body: <p>I am creating a chatbot trained on <a href="https://www.cs.cornell.edu/~cristian/Cornell_Movie-Dialogs_Corpus.html" rel="noreferrer">Cornell Movie Dialogs Corpus</a> using <a href="https://github.com/tensorflow/nmt" rel="noreferrer">NMT</a>.</p>
  
  <p>I am basing my code in part from <a href="https://github.com/bshao001/ChatLearner" rel="noreferrer">https://github.com/bshao001/ChatLearner</a> and <a href="https://github.com/chiphuyen/stanford-tensorflow-tutorials/tree/master/assignments/chatbot" rel="noreferrer">https://github.com/chiphuyen/stanford-tensorflow-tutorials/tree/master/assignments/chatbot</a></p>
  
  <p>During training, I print a random output answer fed to the decoder from the batch and the corresponding answer that my model predicts to observe the learning progress.</p>
  
  <p><strong>My issue:</strong> After only about 4 iterations of training, the model learns to output the EOS token (<code>&lt;\s&gt;</code>) for every timestep. It always outputs that as its response (determined using argmax of logits) even as training continues. Once in a while, rarely, the model outputs series of periods as its answer. </p>
  
  <p>I also print the top 10 logit values during training (not just the argmax) to see if maybe the correct word is somewhere in there, but it seems to be predicting the most common words in the vocab (e.g i, you, ?, .). Even these top 10 words don't change much during training. </p>
  
  <p>I have made sure to correctly count input sequence lengths for encoder and decoder, and added SOS (<code>&lt;s&gt;</code>) and EOS (also used for padding) tokens accordingly. I also perform <strong>masking</strong> in the loss calculation.</p>
  
  <p>Here is a sample output:</p>
  
  <p><strong>Training iteration 1:</strong></p>
  
  <pre><code>Decoder Input: &lt;s&gt; sure . sure . &lt;\s&gt; &lt;\s&gt; &lt;\s&gt; &lt;\s&gt; &lt;\s&gt; &lt;\s&gt; &lt;\s&gt; 
  &lt;\s&gt; &lt;\s&gt;
  Predicted Answer: wildlife bakery mentality mentality administration 
  administration winston winston winston magazines magazines magazines 
  magazines
  </code></pre>
  
  <p>...</p>
  
  <p><strong>Training iteration 4:</strong></p>
  
  <pre><code>Decoder Input: &lt;s&gt; i guess i had it coming . let us call it settled . 
  &lt;\s&gt; &lt;\s&gt; &lt;\s&gt; &lt;\s&gt; &lt;\s&gt;
  Predicted Answer: &lt;\s&gt; &lt;\s&gt; &lt;\s&gt; &lt;\s&gt; &lt;\s&gt; &lt;\s&gt; &lt;\s&gt; &lt;\s&gt; &lt;\s&gt; &lt;\s&gt; 
  &lt;\s&gt; &lt;\s&gt; &lt;\s&gt; &lt;\s&gt; &lt;\s&gt; &lt;\s&gt; &lt;\s&gt; &lt;\s&gt;
  </code></pre>
  
  <p><br>
  After a few more iterations, it settles on only predicting EOS (and rarely some periods)</p>
  
  <p>I am not sure what could be causing this issue and have been stuck on this for a while. Any help would be greatly appreciated!</p>
  
  <p><strong>Update:</strong> I let it train for over a hundred thousand iterations and it still only outputs EOS (and occasional periods). The training loss also does not decrease after a few iteration (it remains at around 47 from the beginning)</p>