My Keras CNN doesn't learn I am experimenting with Keras and I managed to build a simple CNN to classify blue images (300x300 images of the same shade of blue) vs. red images (same size, just red). This is a dummy problem I assumed the NN would have solved immediately but this doesn't seem the case. In fact, even after 20+ epochs accuracy is still exactly 50%.

I assume there's a lot of stuff I could try to do differently but is there anything extraordinary wrong I am doing here that could result in such poor performance on such an easy task?

# Create a Keras model.
model = keras.Sequential()
model.add(
    keras.layers.Conv2D(
        input_shape=(300, 300, 3),
        filters=64,
        kernel_size=(3, 3),
        activation='relu',
    )
)
model.add(
    keras.layers.Conv2D(
        filters=64,
        kernel_size=(3, 3),
        activation='relu',
    )
)
model.add(
    keras.layers.MaxPooling2D(
        pool_size=(2, 2),
        strides=(2, 2),
    )
)
model.add(keras.layers.Flatten())
model.add(
    keras.layers.Dense(
        units=128,
        activation='relu'
    )
)
model.add(keras.layers.Dropout(0.5))
model.add(
    keras.layers.Dense(
        units=1,
        activation='sigmoid',
    )
)

# Train the model.
sgd = keras.optimizers.SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(
    optimizer=sgd,
    loss='mean_squared_error',
    metrics=['binary_accuracy']
)
model.fit(images, labels, epochs=20, batch_size=5)


Which outputs:

20/20 [==============================] - 18s 887ms/step - loss: 0.5954 - binary_accuracy: 0.4000
Epoch 2/20
20/20 [==============================] - 16s 794ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 3/20
20/20 [==============================] - 16s 781ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 4/20
20/20 [==============================] - 17s 853ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 5/20
20/20 [==============================] - 18s 877ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 6/20
20/20 [==============================] - 18s 891ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 7/20
20/20 [==============================] - 17s 825ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 8/20
20/20 [==============================] - 17s 861ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 9/20
20/20 [==============================] - 17s 846ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 10/20
20/20 [==============================] - 17s 835ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 11/20
20/20 [==============================] - 16s 800ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 12/20
20/20 [==============================] - 16s 806ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 13/20
20/20 [==============================] - 16s 811ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 14/20
20/20 [==============================] - 17s 827ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 15/20
20/20 [==============================] - 16s 806ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 16/20
20/20 [==============================] - 16s 786ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 17/20
20/20 [==============================] - 16s 795ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 18/20
20/20 [==============================] - 16s 796ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 19/20
20/20 [==============================] - 16s 788ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 20/20
20/20 [==============================] - 16s 794ms/step - loss: 0.5000 - binary_accuracy: 0.5000
If you remember the theory, CNN kernels basically work on the principle of edge detection. So now that your picture is purely blue or purely red there are no edges. I am guessing that is the main problem here, since I think ML libraries initialise kernels which are suitable for edge detection.
To be more comprehensive:


Since kernels are randomised small numbers, sum of red kernel is almost equal to sum of  blue kernel. The next layer will mainly have (sum of red kernel * 255) and (sum of blue kernel * 255) in all the channels for the next layer, which will be roughly equal. Since you are using an inappropriate loss function, the CNN becomes insensitive to small changes due to the difference in summation of kernels.


I would suggest:


Changing the padding to same might work since then edges will be introduced in the picture (black-red, black - blue). 
Try by adding significant white edges to the side of the picture. 
Squared error does not work that good for sigmoid activations, you need to use the cross entropy loss.
One must i believe experimenting with smaller learning rates, since CNN have the problem of exploding gradients.