title: Explosion in loss function, LSTM autoencoder
body: <p>I am training a LSTM autoencoder, but the loss function randomly shoots up as in the picture below:
  <a href="https://i.stack.imgur.com/enB6N.png" rel="noreferrer"><img src="https://i.stack.imgur.com/enB6N.png" alt="screenshot of explosion in loss function"></a>
  I tried multiple to things to prevent this, adjusting the batch size, adjusting the number of neurons in my layers, but nothing seems to help. I checked my input data to see if it contains null / infinity values, but it doesn't, it is normalized also. Here is my code for reference:</p>
  
  <pre><code>model = Sequential()
  model.add(Masking(mask_value=0, input_shape=(430, 3)))
  model.add(LSTM(100, dropout=0.2, recurrent_dropout=0.2, activation='relu'))
  model.add(RepeatVector(430))
  model.add(LSTM(100, dropout=0.2, recurrent_dropout=0.2, activation='relu', return_sequences=True))
  model.add(TimeDistributed(Dense(3)))
  model.compile(optimizer='adam', loss='mean_squared_error', metrics=['accuracy'])
  
  context_paths = loadFile()
  X_train, X_test = train_test_split(context_paths, test_size=0.20)
  
  history = model.fit(X_train, X_train, epochs=1, batch_size=4, verbose=1, validation_data=(X_test, X_test))
  </code></pre>
  
  <p>The loss function explodes at random points in time, sometimes sooner, sometimes later. I read <a href="https://github.com/keras-team/keras/issues/2134" rel="noreferrer">this</a> thread about possible problems, but at this point after trying multiple things I am not sure what to do to prevent the loss function from skyrocketing at random. Any advice is appreciated. Other than this I can see that my accuracy is not increasing very much, so the problems may be interconnected.</p>