title: Validation accuracy fluctuating while training accuracy increase?
body: <p>I have a multiclassification problem that depends on historical data. I am trying LSTM using loss='sparse_categorical_crossentropy'. The train accuracy and loss increase and decrease respectively. But, my test accuracy starts to fluctuate wildly.</p>
  
  <p>What I am doing wrong?</p>
  
  <p><strong>Input data:</strong></p>
  
  <pre><code>X = np.reshape(X, (X.shape[0], X.shape[1], 1))
  X.shape
  (200146, 13, 1)
  </code></pre>
  
  <p><strong>My model</strong></p>
  
  <pre><code># fix random seed for reproducibility
  seed = 7
  np.random.seed(seed)
  
  # define 10-fold cross validation test harness
  kfold = StratifiedKFold(n_splits=10, shuffle=False, random_state=seed)
  cvscores = []
  for train, test in kfold.split(X, y):
      regressor = Sequential()
  
      # Units = the number of LSTM that we want to have in this first layer -&gt; we want very high dimentionality, we need high number
      # return_sequences =  True because we are adding another layer after this
      # input shape = the last two dimensions and the indicator
      regressor.add(LSTM(units=50, return_sequences=True, input_shape=(X[train].shape[1], 1)))
      regressor.add(Dropout(0.2))
  
      # Extra LSTM layer
      regressor.add(LSTM(units=50, return_sequences=True))
      regressor.add(Dropout(0.2))
      # 3rd
      regressor.add(LSTM(units=50, return_sequences=True))
      regressor.add(Dropout(0.2))
  
      #4th
      regressor.add(LSTM(units=50))
      regressor.add(Dropout(0.2))
  
      # output layer
      regressor.add(Dense(4, activation='softmax', kernel_regularizer=regularizers.l2(0.001)))
  
      # Compile the RNN
      regressor.compile(optimizer='adam', loss='sparse_categorical_crossentropy',metrics=['accuracy'])
  
      # Set callback functions to early stop training and save the best model so far
      callbacks = [EarlyStopping(monitor='val_loss', patience=9),
               ModelCheckpoint(filepath='best_model.h5', monitor='val_loss', save_best_only=True)]
  
  
      history = regressor.fit(X[train], y[train], epochs=250, callbacks=callbacks, 
                          validation_data=(X[test], y[test]))
  
      # plot train and validation loss
      pyplot.plot(history.history['loss'])
      pyplot.plot(history.history['val_loss'])
      pyplot.title('model train vs validation loss')
      pyplot.ylabel('loss')
      pyplot.xlabel('epoch')
      pyplot.legend(['train', 'validation'], loc='upper right')
      pyplot.show()
  
  
      # evaluate the model
      scores = regressor.evaluate(X[test], y[test], verbose=0)
      print("%s: %.2f%%" % (regressor.metrics_names[1], scores[1]*100))
      cvscores.append(scores[1] * 100)
  print("%.2f%% (+/- %.2f%%)" % (np.mean(cvscores), np.std(cvscores)))
  </code></pre>
  
  <p><strong>Results:</strong></p>
  
  <p><a href="https://i.stack.imgur.com/pgbvl.png" rel="nofollow noreferrer">trainingmodel</a></p>
  
  <p><a href="https://i.stack.imgur.com/BEQSG.png" rel="nofollow noreferrer">Plot</a></p>