title: Data Augmentation hurts accuracy Keras
body: <p>I'm trying to adapt <strong>Deep Learning with Python</strong> <em>section 5.3 Feature extraction with Data Augmentation</em> to a 3-class problem with resnet50 (imagenet weights).</p>
  
  <p>Full code at <a href="https://github.com/morenoh149/plantdisease" rel="nofollow noreferrer">https://github.com/morenoh149/plantdisease</a></p>
  
  <pre><code>from keras import models
  from keras import layers
  from keras.applications.resnet50 import ResNet50
  from keras import optimizers
  from keras.preprocessing.image import ImageDataGenerator
  
  input_shape = (224, 224, 3)
  target_size = (224, 224)
  batch_size = 20
  
  conv_base = ResNet50(weights='imagenet', input_shape=input_shape, include_top=False)
  
  model = models.Sequential()
  model.add(conv_base)
  model.add(layers.Flatten())
  model.add(layers.Dense(256, activation='relu'))
  model.add(layers.Dense(3, activation='softmax'))
  
  conv_base.trainable = False
  
  train_datagen = ImageDataGenerator(rescale=1./255)
  test_datagen = ImageDataGenerator(rescale=1./255)
  train_generator = train_datagen.flow_from_directory(
      'input/train',
      target_size=target_size,
      batch_size=batch_size,
      class_mode='categorical')
  validation_generator = test_datagen.flow_from_directory(
      'input/validation',
      target_size=target_size,
      batch_size=batch_size,
      class_mode='categorical')
  model.compile(loss='categorical_crossentropy',
               optimizer=optimizers.RMSprop(lr=2e-5),
               metrics=['acc'])
  history = model.fit_generator(
      train_generator,
      steps_per_epoch=96,
      epochs=30,
      verbose=2,
      validation_data=validation_generator,
      validation_steps=48)
  </code></pre>
  
  <p>Questions:</p>
  
  <ul>
  <li>the book doesn't go much into ImageDataGenerator and selecting <code>steps_per_epoch</code> and <code>validation_steps</code>. What should these values be?
  I have 3 classes, 1000 images each. I've split it 60/20/20 train/validation/test.</li>
  <li>I was able to get a validation accuracy of 60% without data augmentation. Above I've simplified the ImageDataGenerator to only rescale. This model has a validation accuracy of 30% Why?</li>
  <li>What changes do I need to make to the data-augmented version of this script to match the accuracy with no augmentation?</li>
  </ul>
  
  <p>UPDATE:
  This may be an issue with keras itself</p>
  
  <ul>
  <li><a href="https://github.com/keras-team/keras/issues/9214" rel="nofollow noreferrer">https://github.com/keras-team/keras/issues/9214</a></li>
  <li><a href="https://github.com/keras-team/keras/pull/9965" rel="nofollow noreferrer">https://github.com/keras-team/keras/pull/9965</a></li>
  </ul>