title: transfer learning of Inception v3 returns the same predictions
body: <p>I have tried to customize inception to classify. I used the cat, dog, human and other to classify cat, dog, human ( collection of family photos) and other (mostly natural scenes) I have about 2000 dog, 1000 cat, 7000 human and 3000 images split 80:20 among the train and validate. The essence of model is as below. When I train the training accuracy is close to 97% and validation accuracy is ~90%.</p>
  <pre><code>    import os
  
      from tensorflow.keras import layers
      from tensorflow.keras import Model
      from tensorflow.keras.optimizers import RMSprop
        
      from tensorflow.keras.applications.inception_v3 import InceptionV3
      
      local_weights_file = 'C:/users/sethr/education/Healthcare/imagedetect/modelimageadvance /inception_v3_weights_tf_dim_ordering_tf_kernels_notop.h5'
      
      pre_trained_model = InceptionV3(input_shape = (150, 150,3), include_top = False, weights =  `enter code here`local_weights_file)
      for layer in pre_trained_model.layers:
          layer.trainable = False 
      # pre_trained_model.summary()
      last_layer = pre_trained_model.get_layer('mixed7')
      #print('last layer output shape: ', last_layer.output_shape)
      last_output = last_layer.output
      # Flatten the output layer to 1 dimension
      x = tf.keras.layers.Flatten()(last_output)
      # Add a fully connected layer with 1,024 hidden units and ReLU activation
      x = tf.keras.layers.Dense(512, activation='relu')(x)
      x = tf.keras.layers.Dense(256, activation='relu')(x)
      # Add a dropout rate of 0.2
      x = tf.keras.layers.Dropout(0.2)(x)                  
      # Add a final sigmoid layer for classification
      x = tf.keras.layers.Dense(4, activation='softmax')(x)           
      
      model = Model(pre_trained_model.input, x)
      
      model.compile(optimizer = RMSprop(lr=0.0001), loss = 'categorical_crossentropy', metrics = ['accuracy'])
      
      history = model.fit(train_generator,validation_data = validation_generator,steps_per_epoch = 20,epochs = 20,validation_steps = 25,verbose = 2)
      model.save(&quot;dcho.rp5&quot;)
      ___________________
      
      import numpy as np
      import cv2
      import glob
      from keras.preprocessing import image
      import matplotlib.image as mpimg
      import matplotlib.pyplot as plt
      
      labels= [&quot;cat&quot;,&quot;dog&quot;,&quot;human&quot;,&quot;other&quot;]
      path='C:/Users/sethr/education/Healthcare/imagedetect/images/*.jpg'
      for fim in glob.glob(path):
      
        # predicting images
        img=image.load_img(fim, target_size=(150, 150))
        
        x=image.img_to_array(img)
        x=np.expand_dims(x, axis=0)
        images = np.vstack([x])
        plt.figure()
        plt.imshow(img)
        plt.show()
        classes = model.predict(images,batch_size=10)
  
  ________________________________________________________
  
  _______________________________________________________
  
  
  # All images will be rescaled by 1./255.
  train_datagen = ImageDataGenerator( rescale = 1.0/255. )
  test_datagen  = ImageDataGenerator( rescale = 1.0/255. )
  
  # --------------------
  # Flow training images in batches of 20 using train_datagen generator
  # -------------------- 
  train_generator = train_datagen.flow_from_directory(train_dir,
                                                      batch_size=20,
                                                      shuffle='True',
                                                      class_mode='categorical', 
                                                      target_size=(150, 150)) 
  # --------------------
  
      # Flow validation images in batches of 20 using test_datagen generator
      # --------------------
      validation_generator =  test_datagen.flow_from_directory(validation_dir,
                                                               target_size = (150, 150),
                                                               batch_size=20,
                                                               class_mode='categorical',
                                                               shuffle='True',
                                                               )
  
  _______________________________________________________________
  </code></pre>
  <p>The problem is it is returning always human as prediction.. I played around with Adam, Adjusted learning rate but still the prediction remains the same. Any insight.</p>