Deep Learning for high contrast images with small differences
I have solved my problem! I found a relevant article which describes an architecture suitable for my problem. I have changed it a bit however. The article: https://www.researchgate.net/publication/312328148_Binary_shape_classification_using_Convolutional_Neural_Networks. 

The model in keras:

modelBS = Sequential()

modelBS.add(Conv2D(150, (3, 3), input_shape=(ImgSize,ImgSize,1), activation='relu', padding='same'))
modelBS.add(MaxPooling2D(pool_size=(5, 5), strides=(5, 5)))

modelBS.add(Conv2D(150, (5, 5), activation='relu', padding='valid'))
modelBS.add(Conv2D(150, (2, 2), activation='relu', padding='valid'))

modelBS.add(Flatten())
modelBS.add(Dense(rp, activation='softmax'))