title: Pytorch RNN prediction results do not match training results
body: <p>The RNN code produces good sine approximations during traing, but when it comes to predicting sine values based in previous predictions (with a head start of 20 original values), it fails to approximate sine. (check images <a href="https://imgur.com/a/GCQ6QgG" rel="nofollow noreferrer">https://imgur.com/a/GCQ6QgG</a>)</p>
  
  <p>So either there is a bug in the prediction code or the model in general is not capable of handling refeeded predictions. But I think its more of a bug in my code :)</p>
  
  <p>Code:
  <a href="https://gist.github.com/saschalippert/36b72313afff86e00f3e10254fb4ff25" rel="nofollow noreferrer">https://gist.github.com/saschalippert/36b72313afff86e00f3e10254fb4ff25</a></p>
  
  <p>Logs:   </p>
  
  <pre><code>RNN(
    (rnn): RNN(1, 64, batch_first=True)
    (fc): Linear(in_features=64, out_features=1, bias=True)
  )
  Training for 10 epoch(s)...
  Epoch:    1/10    Loss: 0.05084234103560448
  Epoch:    2/10    Loss: 0.006296889390796423
  Epoch:    3/10    Loss: 0.005784639157354832
  Epoch:    4/10    Loss: 0.0037908838130533695
  Epoch:    5/10    Loss: 0.0030788308940827847
  Epoch:    6/10    Loss: 0.0027744979597628117
  Epoch:    7/10    Loss: 0.0027218328323215246
  Epoch:    8/10    Loss: 0.00589225348085165
  Epoch:    9/10    Loss: 0.0020567490719258785
  Epoch:   10/10    Loss: 0.000807589094620198
  </code></pre>