My Keras CNN doesn't learn
If you remember the theory, CNN kernels basically work on the principle of edge detection. So now that your picture is purely blue or purely red there are no edges. I am guessing that is the main problem here, since I think ML libraries initialise kernels which are suitable for edge detection.
To be more comprehensive:


Since kernels are randomised small numbers, sum of red kernel is almost equal to sum of  blue kernel. The next layer will mainly have (sum of red kernel * 255) and (sum of blue kernel * 255) in all the channels for the next layer, which will be roughly equal. Since you are using an inappropriate loss function, the CNN becomes insensitive to small changes due to the difference in summation of kernels.


I would suggest:


Changing the padding to same might work since then edges will be introduced in the picture (black-red, black - blue). 
Try by adding significant white edges to the side of the picture. 
Squared error does not work that good for sigmoid activations, you need to use the cross entropy loss.
One must i believe experimenting with smaller learning rates, since CNN have the problem of exploding gradients.