title: LSTM model for time series predictions predicts irregular values like a sawtooth
body: <p>I am training a Keras model to predict availability of bike-sharing stations. I am giving in the training set a whole row with day of the year, time, weekday, station and free bikes. Each sample contains the availability for the previous day (144 samples) and I am trying to predict the availability for the next day (144 samples). The shapes for the sets used are</p>
  
  <pre><code>Train X (2362, 144, 5)
  Train Y (2362, 144)
  Test X (39, 144, 5)
  Test Y (39, 144)
  Validation X (1535, 144, 5)
  Validation Y (1535, 144)
  </code></pre>
  
  <p>The model I am using is this one</p>
  
  <pre><code>    model.add(LSTM(20, input_shape=(self.train_x.shape[1], self.train_x.shape[2]), return_sequences = True))
      model.add(Dropout(0.2))
      model.add(LSTM(20))
      model.add(Dense(144))
      model.compile(loss='mse', optimizer='adam', metrics = ['acc', 'mape', 'mse'])
  
      history = self.model.fit(self.train_x, self.train_y, batch_size=50, epochs=20, validation_data=(self.validation_x, self.validation_y), verbose=1, shuffle = True)
  </code></pre>
  
  <p>The predictions made after training have nothing to do with the expected output, they have like a sawtooth shape with values that exceed the original size.</p>
  
  <p><a href="https://i.stack.imgur.com/G0ivi.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/G0ivi.png" alt="Prediction Sample"></a></p>
  
  <p>The accuracy rarely goes up but loss has a normal shape.</p>
  
  <p><a href="https://i.stack.imgur.com/XLnt1.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/XLnt1.png" alt="Accuracy"></a></p>
  
  <p><a href="https://i.stack.imgur.com/jH5Ph.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/jH5Ph.png" alt="Loss"></a></p>
  
  <p>As an example the history after each epoch looks like this</p>
  
  <pre><code>Epoch 17/20
  2362/2362 [==============================] - 12s 5ms/step - loss: 9.1214 - acc: 0.0000e+00 - mean_absolute_percentage_error: 21925846.0813 - mean_squared_error: 9.1214 - val_loss: 9.0642 - val_acc: 0.0000e+00 - val_mean_absolute_percentage_error: 24162847.3779 - val_mean_squared_error: 9.0642
  Epoch 18/20
  2362/2362 [==============================] - 12s 5ms/step - loss: 8.2241 - acc: 0.0013 - mean_absolute_percentage_error: 21906919.9136 - mean_squared_error: 8.2241 - val_loss: 8.1923 - val_acc: 0.0000e+00 - val_mean_absolute_percentage_error: 22754663.8013 - val_mean_squared_error: 8.1923
  Epoch 19/20
  2362/2362 [==============================] - 12s 5ms/step - loss: 7.4190 - acc: 0.0000e+00 - mean_absolute_percentage_error: 21910003.1744 - mean_squared_error: 7.4190 - val_loss: 7.3926 - val_acc: 0.0000e+00 - val_mean_absolute_percentage_error: 24673277.8420 - val_mean_squared_error: 7.3926
  Epoch 20/20
  2362/2362 [==============================] - 12s 5ms/step - loss: 6.7067 - acc: 0.0013 - mean_absolute_percentage_error: 22076339.2168 - mean_squared_error: 6.7067 - val_loss: 6.6758 - val_acc: 6.5147e-04 - val_mean_absolute_percentage_error: 22987089.8436 - val_mean_squared_error: 6.6758
  </code></pre>
  
  <p>I really don't know where the problem might be, more layers?, less layers?, different approach?</p>
  
  <p><strong>UPDATE: Plots of training/test data</strong>. Left part of the plot shows the previous day of availability that is fed to the model and the right part shows what the result should be and the prediction made by the model.</p>
  
  <p><a href="https://i.stack.imgur.com/FuEIb.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/FuEIb.png" alt="Predicted"></a></p>