title: CNN with keras, accuracy not improving
body: <p>I have started with Machine Learning recently, I am learning CNN, I planned to write an application for Car Damage severity detection, with the help of this <a href="https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html" rel="noreferrer">Keras blog</a> and this <a href="https://github.com/neokt/car-damage-detective/blob/master/04-classifying_damage_severity.ipynb" rel="noreferrer">github repo</a>. </p>
  
  <p>This is how car data-set looks like:</p>
  
  <pre><code>F:\WORKSPACE\ML\CAR_DAMAGE_DETECTOR\DATASET\DATA3A
  ├───training (979 Images for all 3 categories of training set)
  │   ├───01-minor
  │   ├───02-moderate
  │   └───03-severe
  └───validation (171 Images for all 3 categories of validation set)
      ├───01-minor
      ├───02-moderate
      └───03-severe
  </code></pre>
  
  <p>Following code gives me only 32% of accuracy.</p>
  
  <pre><code>from keras.preprocessing.image import ImageDataGenerator
  from keras.models import Sequential
  from keras.layers import Conv2D, MaxPooling2D
  from keras.layers import Activation, Dropout, Flatten, Dense
  from keras import backend as K
  
  
  # dimensions of our images.
  img_width, img_height = 150, 150
  
  train_data_dir = 'dataset/data3a/training'
  validation_data_dir = 'dataset/data3a/validation'
  nb_train_samples = 979
  nb_validation_samples = 171
  epochs = 10
  batch_size = 16
  
  if K.image_data_format() == 'channels_first':
      input_shape = (3, img_width, img_height)
  else:
      input_shape = (img_width, img_height, 3)
  
  model = Sequential()
  model.add(Conv2D(32, (3, 3), input_shape=input_shape))
  model.add(Activation('relu'))
  model.add(MaxPooling2D(pool_size=(2, 2)))
  
  model.add(Conv2D(32, (3, 3)))
  model.add(Activation('relu'))
  model.add(MaxPooling2D(pool_size=(2, 2)))
  
  model.add(Conv2D(64, (3, 3)))
  model.add(Activation('relu'))
  model.add(MaxPooling2D(pool_size=(2, 2)))
  
  model.add(Flatten())
  model.add(Dense(64))
  model.add(Activation('relu'))
  model.add(Dropout(0.5))
  model.add(Dense(1))
  model.add(Activation('sigmoid'))
  
  model.compile(loss='binary_crossentropy',
                optimizer='rmsprop',
                metrics=['accuracy'])
  
  # this is the augmentation configuration we will use for training
  train_datagen = ImageDataGenerator(
      rescale=1. / 255,
      shear_range=0.2,
      zoom_range=0.2,
      horizontal_flip=True)
  
  # this is the augmentation configuration we will use for testing:
  # only rescaling
  test_datagen = ImageDataGenerator(rescale=1. / 255)
  
  train_generator = train_datagen.flow_from_directory(
      train_data_dir,
      target_size=(img_width, img_height),
      batch_size=batch_size,
      class_mode='binary')
  
  validation_generator = test_datagen.flow_from_directory(
      validation_data_dir,
      target_size=(img_width, img_height),
      batch_size=batch_size,
      class_mode='binary')
  model.fit_generator(
      train_generator,
      steps_per_epoch=nb_train_samples // batch_size,
      epochs=epochs,
      validation_data=validation_generator,
      validation_steps=nb_validation_samples // batch_size)
  
  model.save_weights('first_try.h5')
  </code></pre>
  
  <p>I tried:</p>
  
  <ul>
  <li>By increasing the epochs to 10, 20,50.</li>
  <li>By increasing images in the dataset (all validation images added to training set).</li>
  <li>By updating the filter size in the <code>Conv2D</code> layer</li>
  <li>Tried to add couple of <code>Conv2D</code> layer, <code>MaxPooling</code> layers</li>
  <li>Also tried with different optimizers such as <code>adam</code>, <code>Sgd</code>, etc</li>
  <li>Also Tried by updating the filter strides to <code>(1,1) and (5,5)</code> instead of <code>(3,3)</code></li>
  <li>Also tried by updating the changing image dimensions to <code>(256, 256)</code>, <code>(64, 64)</code> from <code>(150, 150)</code> </li>
  </ul>
  
  <p>But no luck, every-time I'm getting accuracy up to 32% or less than that but not more.
  Any idea what I'm missing.</p>
  
  <p>As in the <a href="https://github.com/neokt/car-damage-detective/blob/master/04-classifying_damage_severity.ipynb" rel="noreferrer">github repo</a> we can see,  it gives 72% accuracy for the same dataset (Training -979, Validation -171). Why its not working for me.</p>
  
  <p>I tried his code from the github link on my machine but it hanged up while training the dataset(I waited for more than 8 hours), so changed the approach, but still no luck so far.</p>
  
  <p>Here's the <a href="https://pastebin.com/raw/dABSdNmR" rel="noreferrer">Pastebin</a> containing output of my training epochs.</p>