title: Loss not changing no matter the learning rate
body: <p>I have built a deep learning model, slightly similar to the VGG net. I'm using Keras with a Tensorflow backend. The model summary is as follows:</p>
  
  <pre><code>model = Sequential()
  model.add(Conv2D(64, 3, border_mode='same', activation='relu', input_shape=(180,320,3)))
  model.add(Conv2D(64, 3, border_mode='same', activation='relu'))
  model.add(MaxPooling2D(pool_size=(2,2), strides=2))
  model.add(Conv2D(64, 3, border_mode='same', activation='relu'))
  model.add(Conv2D(64, 3, border_mode='same', activation='relu'))
  model.add(MaxPooling2D(pool_size=(2,2), strides=2))
  model.add(Conv2D(128, 3, border_mode='same', activation='relu'))
  model.add(Conv2D(128, 3, border_mode='same', activation='relu'))
  model.add(MaxPooling2D(pool_size=(2,2), strides=2))
  model.add(Conv2D(128, 3, border_mode='same', activation='relu'))
  model.add(Conv2D(128, 3, border_mode='same', activation='relu'))
  model.add(Flatten())
  model.add(Dropout(0.5))
  model.add(Dense(1024, activation='relu'))
  model.add(Dropout(0.5))
  model.add(Dense(9,  activation='relu'))
  </code></pre>
  
  <p>I have tried different combinations of optimizer (SGD, Adam, etc.), loss (MSE, MAE, etc.), batch sizes (32 and 64). I have even experimented with learning rates ranging from 0.001 to 10000. But still, even after 20 epochs, for whichever loss function I use, the validation loss remains exactly the same. The training loss changes insignificantly. What am I doing wrong? </p>
  
  <p>What my network should be trained to do: Given an input image, the network needs to predict a set of 9 real values which can be derived from this image. </p>
  
  <p>Terminal output during training: </p>
  
  <pre><code>    Epoch 1/100
      4800/4800 [==============================] - 96s 20ms/step - loss: 133.6534 - mean_absolute_error: 133.6534 - val_loss: 132.3744 - val_mean_absolute_error: 132.3744
      Epoch 2/100
      4800/4800 [==============================] - 49s 10ms/step - loss: 132.8033 - mean_absolute_error: 132.8033 - val_loss: 132.3744 - val_mean_absolute_error: 132.3744
      Epoch 3/100
      4800/4800 [==============================] - 51s 11ms/step - loss: 132.8033 - mean_absolute_error: 132.8033 - val_loss: 132.3744 - val_mean_absolute_error: 132.3744
      Epoch 4/100
      4800/4800 [==============================] - 51s 11ms/step - loss: 132.8033 - mean_absolute_error: 132.8033 - val_loss: 132.3744 - val_mean_absolute_error: 132.3744
      Epoch 5/100
      4800/4800 [==============================] - 51s 11ms/step - loss: 132.8033 - mean_absolute_error: 132.8033 - val_loss: 132.3744 - val_mean_absolute_error: 132.3744
      Epoch 6/100
      4800/4800 [==============================] - 51s 11ms/step - loss: 132.8033 - mean_absolute_error: 132.8033 - val_loss: 132.3744 - val_mean_absolute_error: 132.3744
      Epoch 7/100
      4800/4800 [==============================] - 51s 11ms/step - loss: 132.8033 - mean_absolute_error: 132.8033 - val_loss: 132.3744 - val_mean_absolute_error: 132.3744
      Epoch 8/100
      4800/4800 [==============================] - 51s 11ms/step - loss: 132.8033 - mean_absolute_error: 132.8033 - val_loss: 132.3744 - val_mean_absolute_error: 132.3744
      Epoch 9/100
      4800/4800 [==============================] - 51s 11ms/step - loss: 132.8033 - mean_absolute_error: 132.8033 - val_loss: 132.3744 - val_mean_absolute_error: 132.3744
      Epoch 10/100
      4800/4800 [==============================] - 51s 11ms/step - loss: 132.8033 - mean_absolute_error: 132.8033 - val_loss: 132.3744 - val_mean_absolute_error: 132.3744
      Epoch 11/100
      4800/4800 [==============================] - 51s 11ms/step - loss: 132.8033 - mean_absolute_error: 132.8033 - val_loss: 132.3744 - val_mean_absolute_error: 132.3744
      Epoch 12/100
      4800/4800 [==============================] - 51s 11ms/step - loss: 132.8033 - mean_absolute_error: 132.8033 - val_loss: 132.3744 - val_mean_absolute_error: 132.3744
      Epoch 13/100
      4800/4800 [==============================] - 50s 10ms/step - loss: 132.8033 - mean_absolute_error: 132.8033 - val_loss: 132.3744 - val_mean_absolute_error: 132.3744
      Epoch 14/100
      4800/4800 [==============================] - 51s 11ms/step - loss: 132.8033 - mean_absolute_error: 132.8033 - val_loss: 132.3744 - val_mean_absolute_error: 132.3744
      Epoch 15/100
      4800/4800 [==============================] - 51s 11ms/step - loss: 132.8033 - mean_absolute_error: 132.8033 - val_loss: 132.3744 - val_mean_absolute_error: 132.3744
      Epoch 16/100
      4800/4800 [==============================] - 51s 11ms/step - loss: 132.8033 - mean_absolute_error: 132.8033 - val_loss: 132.3744 - val_mean_absolute_error: 132.3744
      Epoch 17/100
      4800/4800 [==============================] - 51s 11ms/step - loss: 132.8033 - mean_absolute_error: 132.8033 - val_loss: 132.3744 - val_mean_absolute_error: 132.3744
      Epoch 18/100
      4800/4800 [==============================] - 51s 11ms/step - loss: 132.8033 - mean_absolute_error: 132.8033 - val_loss: 132.3744 - val_mean_absolute_error: 132.3744
      Epoch 19/100
      4800/4800 [==============================] - 51s 11ms/step - loss: 132.8033 - mean_absolute_error: 132.8033 - val_loss: 132.3744 - val_mean_absolute_error: 132.3744
      Epoch 20/100
      4800/4800 [==============================] - 51s 11ms/step - loss: 132.8033 - mean_absolute_error: 132.8033 - val_loss: 132.3744 - val_mean_absolute_error: 132.3744
      Epoch 21/100
      4800/4800 [==============================] - 50s 10ms/step - loss: 132.8033 - mean_absolute_error: 132.8033 - val_loss: 132.3744 - val_mean_absolute_error: 132.3744
  </code></pre>