title: High diversity between val_accuracy and evaluation results
body: <p>I'm working on a multiclass text classification problem.</p>
  <p>After splitting the data to train and validation data frames, I've performed text augmentation to balance the data (only on the train data of course).</p>
  <p>I've ended up with a balanced trained data and 44325 samples (of trained data).
  Later on I've applied the &quot;clean text&quot; task for getting (i.e. stemming and stuff) on the trained data.</p>
  <pre><code>train['text'] = train['text'].apply(clean_text)
  X_train = train.iloc[:, :-1]
  y_train = train.iloc[:, -1:]
  X_test = valid.iloc[:, :-1]
  y_test = valid.iloc[:, -1:]
  
  y_test = pd.DataFrame(y_test).reset_index(drop=True)
  
  tokenizer = Tokenizer(num_words=vocab_size, oov_token='&lt;OOV&gt;')
  tokenizer.fit_on_texts(X_train['text'])
  
  train_seq = tokenizer.texts_to_sequences(X_train['text'])
  train_padded = pad_sequences(train_seq, maxlen=max_length, padding=padding_type, truncating=trunc_type)
  
  validation_seq = tokenizer.texts_to_sequences(X_test['text'])
  validation_padded = pad_sequences(validation_seq, maxlen=max_length, padding=padding_type, truncating=trunc_type)
  print('Shape of train data tensor:', train_padded.shape)
  print('Shape of validation data tensor:', validation_padded.shape)
  </code></pre>
  <p>Output:</p>
  <pre><code>Shape of train data tensor: (44325, 200)
  Shape of validation data tensor: (5466, 200)
  </code></pre>
  <p>Here's the encoding section:</p>
  <pre><code>encode = OneHotEncoder()
  
  training_labels = encode.fit_transform(y_train)
  validation_labels = encode.transform(y_test)
  
  training_labels = training_labels.toarray()
  validation_labels = validation_labels.toarray()
  </code></pre>
  <p>Model:</p>
  <pre><code>model = Sequential()
  model.add(Embedding(vocab_size, embedding_dim, input_length=train_padded.shape[1]))
  
  model.add(Conv1D(48, len(GROUPS), activation='relu', padding='valid'))
  model.add(GlobalMaxPooling1D())
  model.add(Dropout(0.5))
  
  model.add(Flatten())
  model.add(Dropout(0.5))
  
  model.add(Dense(len(GROUPS), activation='softmax'))
  model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
  
  epochs = 100
  batch_size = 32
  
  history = model.fit(train_padded, training_labels, shuffle=True ,
                      epochs=epochs, batch_size=batch_size, 
                      validation_split=0.2,
                      validation_data=(validation_padded, validation_labels),
                      callbacks=[ReduceLROnPlateau(monitor='val_loss', factor=0.2, patience=5, min_lr=0.0001), 
                                 EarlyStopping(monitor='val_loss', mode='min', patience=2, verbose=1),
                                 EarlyStopping(monitor='val_accuracy', mode='max', patience=5, verbose=1)])
  </code></pre>
  <p>Model output:</p>
  <pre><code>Epoch 1/100
  1109/1109 [==============================] - 88s 79ms/step - loss: 1.2021 - accuracy: 0.5235 - val_loss: 0.8374 - val_accuracy: 0.7232
  Epoch 2/100
  1109/1109 [==============================] - 87s 79ms/step - loss: 0.9505 - accuracy: 0.6645 - val_loss: 0.7488 - val_accuracy: 0.7461
  Epoch 3/100
  1109/1109 [==============================] - 86s 77ms/step - loss: 0.8378 - accuracy: 0.7058 - val_loss: 0.6686 - val_accuracy: 0.7663
  Epoch 4/100
  1109/1109 [==============================] - 88s 79ms/step - loss: 0.7391 - accuracy: 0.7382 - val_loss: 0.6134 - val_accuracy: 0.7891
  Epoch 5/100
  1109/1109 [==============================] - 89s 80ms/step - loss: 0.6763 - accuracy: 0.7546 - val_loss: 0.5832 - val_accuracy: 0.7997
  Epoch 6/100
  1109/1109 [==============================] - 87s 79ms/step - loss: 0.6185 - accuracy: 0.7760 - val_loss: 0.5529 - val_accuracy: 0.8050
  Epoch 7/100
  1109/1109 [==============================] - 87s 79ms/step - loss: 0.5737 - accuracy: 0.7912 - val_loss: 0.5311 - val_accuracy: 0.8153
  Epoch 8/100
  1109/1109 [==============================] - 88s 80ms/step - loss: 0.5226 - accuracy: 0.8080 - val_loss: 0.5268 - val_accuracy: 0.8226
  Epoch 9/100
  1109/1109 [==============================] - 88s 79ms/step - loss: 0.4955 - accuracy: 0.8171 - val_loss: 0.5142 - val_accuracy: 0.8285
  Epoch 10/100
  1109/1109 [==============================] - 88s 80ms/step - loss: 0.4665 - accuracy: 0.8265 - val_loss: 0.5035 - val_accuracy: 0.8338
  Epoch 11/100
  1109/1109 [==============================] - 88s 79ms/step - loss: 0.4410 - accuracy: 0.8348 - val_loss: 0.5082 - val_accuracy: 0.8399
  Epoch 12/100
  1109/1109 [==============================] - 88s 80ms/step - loss: 0.4190 - accuracy: 0.8407 - val_loss: 0.5160 - val_accuracy: 0.8414
  Epoch 00012: early stopping
  </code></pre>
  <p>... and to the last part I'm unsure of:</p>
  <pre><code>def evaluate_preds(y_true, y_preds):
      &quot;&quot;&quot;
      Performs evaluation comparison on y_true labels vs. y_pred labels
      on a classification.
      &quot;&quot;&quot;
      accuracy = accuracy_score(y_true, y_preds)
      precision = precision_score(y_true, y_preds, average='micro')
      recall = recall_score(y_true, y_preds, average='micro')
      f1 = f1_score(y_true, y_preds, average='micro')
      metric_dict = {&quot;accuracy&quot;: round(accuracy, 2),
                     &quot;precision&quot;: round(precision, 2),
                     &quot;recall&quot;: round(recall, 2),
                     &quot;f1&quot;: round(f1, 2)}
      print(f&quot;Acc: {accuracy * 100:.2f}%&quot;)
      print(f&quot;Precision: {precision:.2f}&quot;)
      print(f&quot;Recall: {recall:.2f}&quot;)
      print(f&quot;F1 score: {f1:.2f}&quot;)
      
      return metric_dict
  
  predicted = model.predict(validation_padded)
  evaluate_preds(np.argmax(validation_labels, axis=1), np.argmax(predicted, axis=1))
  </code></pre>
  <p>Output:</p>
  <pre><code>Acc: 40.16%
  Precision: 0.40
  Recall: 0.40
  F1 score: 0.40
  </code></pre>
  <p>I can't understand what am I doing wrong.
  How come accuracy of the last mentioned method is so low compare to val_accuracy?</p>