Implementing the SVHN CNN architecture in Srivastava et al. 2014 Dropout paper
This means that your input shape was too small by the time it reached the last Conv2D layer.

By removing the last Conv2D and MaxPool2D layer, this is how the model looks like:


You can see that the last output before the Flatten layer has height and width of 4, which is smaller than your kernel_size of 5 in the last Conv2D layer.

Heres what you can do:


Increase the input image height and width OR
Set padding='same' in the Conv2D layers