title: LSTM results seems to be shifted by one time period back
body: <p>With LSTM and Keras I built a simple time series prediction model which works like this:
  Providing that I have the data (normalized and prepared for training) for some past 10 time periods, the model predicts the values of the proceeding two time periods.</p>
  
  <p>My Data is shaped similar to this:</p>
  
  <p>X = [[[1,2,3,4,5,6,7,8,9,10],[2,3,4,5,6,7,8,9,10,11],[3,4,5,6,7,8,9,10,11,12]]]</p>
  
  <p>Y = [[11,12],[12,13],[14,15]]</p>
  
  <p>And the model is composed of one LSTM layer with rnn_size number of nodes
  as well as one dropout layer.</p>
  
  <pre><code>    model = Sequential()
  
      model.add(LSTM(rnn_size,
      batch_input_shape=(batch_size, X.shape[1], 
      X.shape[2]),stateful=True,dropout=dropout))
  
      model.add(Dropout(dropout)) 
      model.add(Dense(y.shape[1]))
      adam_optimizer = keras.optimizers.Adam(clipvalue=5)
      model.compile(loss='mean_squared_error', optimizer=adam_optimizer)
      history = model.fit(X, y, batch_size=batch_size, epochs=nb_epoch, 
                      verbose=2, validation_split=0.1,shuffle=False)
      #################################    
      # Predict
      result = scaler.inverse_transform(
                model.predict_on_batch(test_values[start_date:end_date]
                .values.reshape(1, 1, -1)))
  </code></pre>
  
  <p>The problem is that the predicted value for time period t+1 - as can be seen in the graph - seems to be somehow just a slightly different value from the one at the time period t.</p>
  
  <p>Is this a (normal) behaviour?
  If not, how can I overcome it?</p>
  
  <p>I think I have a relatively good understanding of how LSTM work, however I could not wrap my head around this particular issue.</p>
  
  <h2>Edit 1:</h2>
  
  <p>I edited the code to use stateless network configurations and edited the shape as suggested by Daniel. However the problem persist.
  Strangely also, the validation loss (MSE) is always lower than the training loss.</p>
  
  <h2>Edit 2:</h2>
  
  <p>Rest of the code added as per <a href="https://stackoverflow.com/users/5974433/marcin-mo%C5%BCejko">Marcin's</a> request</p>
  
  <p><a href="https://i.stack.imgur.com/WGzbu.png" rel="nofollow noreferrer">actual value vs. predicted value for the test data set that was not shown to the model before</a></p>
  
  <p><a href="https://i.stack.imgur.com/rdkUC.png" rel="nofollow noreferrer">Validation Loss is less than Training Loss! ( small fluctuation is because of Dropout )</a></p>