title: Keras CNN: validation accuracy stuck at 70%, training accuracy reaching 100%
body: <p>I'm trying to create a neural network using Python with Keras, with the purpose of identifying doors and windows on photos and video. Here is my architecture:</p>
  
  <pre><code>img_width = 32
  img_height = 32
  
  model = Sequential()
  model.add(Conv2D(32, (3, 3), padding='same', activation='relu', input_shape=(img_width, img_height, 3)))
  model.add(BatchNormalization())
  model.add(Conv2D(32, (3, 3), activation='relu'))
  model.add(BatchNormalization())
  model.add(MaxPooling2D(pool_size=(2, 2)))
  
  model.add(Conv2D(64, (3, 3), padding='same', activation='relu'))
  model.add(BatchNormalization())
  model.add(Conv2D(64, (3, 3), activation='relu'))
  model.add(BatchNormalization())
  model.add(MaxPooling2D(pool_size=(2, 2)))
  
  model.add(Conv2D(128, (3, 3), padding='same', activation='relu'))
  model.add(BatchNormalization())
  model.add(Conv2D(128, (3, 3), activation='relu'))
  model.add(BatchNormalization())
  model.add(MaxPooling2D(pool_size=(2, 2)))
  
  model.add(Flatten())
  model.add(Dense(256, activation='relu'))
  model.add(Dropout(0.5))
  model.add(Dense(3, activation='softmax'))
  </code></pre>
  
  <p>I'm using the Adam optimizer with default parameters and categorical crossentropy loss function. I've got 3 classes of input data, 685 pictures each and I'm using ImageDataGenerator on them. I've also multiplied the <code>steps_per_epoch</code> and <code>validation_steps</code> values of <code>fit_generator</code> to compensate for the small datasets. Batch size is 32.</p>
  
  <pre><code>train_data_generator = ImageDataGenerator(
      rescale=1. / 255,
      width_shift_range=0.2,
      height_shift_range=0.2,
      zoom_range=0.2,
      horizontal_flip=True,
      validation_split=0.2)
  
  train_generator = train_data_generator.flow_from_directory(
      train_data_dir,
      target_size=(img_width, img_height),
      batch_size=batch_size,
      class_mode='categorical',
      subset='training')
  
  validation_generator = train_data_generator.flow_from_directory(
      train_data_dir,
      target_size=(img_width, img_height),
      batch_size=batch_size,
      class_mode='categorical',
      subset='validation')
  
  early_stopping = EarlyStopping(monitor='val_loss',
                             min_delta=1e-3,
                             patience=10,
                             verbose=1,
                             mode='auto',
                             restore_best_weights=True)
  
  history = model.fit_generator(
      train_generator,
      steps_per_epoch=8*nb_train_samples // batch_size,
      epochs=epochs,
      validation_data=validation_generator,
      validation_steps=8*nb_validation_samples // batch_size,
      callbacks=[tensor_board, model_checkpoint, early_stopping])
  </code></pre>
  
  <p>Now, my problem is the fact that my validation accuracy usually gets stuck at ~70% while the validation loss starts increasing rapidly. At the same time, training loss gets close to 0 while training accuracy almost reaches 100%. 
  So far I've tried to counter it by:</p>
  
  <ul>
  <li>changing batch size to other powers of 2;</li>
  <li>changing Adam's learning rate;</li>
  <li>trying another optimizer;</li>
  <li>employing <code>ReduceLROnPlateau</code> or <code>LearningRateScheduler</code>;</li>
  <li>changing the value of Dropout's parameter to anything in the range 0.2-0.95;</li>
  <li>using Dropouts instead of BatchNormalization;</li>
  <li>changing the size of images.</li>
  </ul>
  
  <p>And of course various combinations of these. I've also changed the entire dataset (before it was inconsistent, the images differed greatly inside the classes and the classes had different sizes). Nothing seemed to work. Any ideas what I might be doing wrong?</p>