title: Why there will be a baseline for LSTM and how to improve the performance?
body: <p>I have a LSTM network with the following configuration:</p>
  
  <pre><code>model3 = tf.keras.Sequential()
  model3.add(tf.keras.layers.LSTM(15, input_shape=(1, 10), return_sequences=True)) 
  model3.add(tf.keras.layers.LSTM(5))
  model3.add(tf.keras.layers.Dense(1, activation=None))
  model3.compile(loss='mean_squared_error', optimizer='adam')
  model3.fit(trainX, trainY, epochs=100, batch_size=30)
  </code></pre>
  
  <p>The result can predict the tendency but there will be a stable baseline, which is abnormal. The figure is like the following: </p>
  
  <p><a href="https://i.stack.imgur.com/iAEZN.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/iAEZN.png" alt="prediction results"></a></p>
  
  <p>So I wonder why there is a stable line even if there is no activation function in my last layer. And also how to improve performance. </p>