Why does my Keras model learn to recognize the background?
Your model is overfitting. Each epoch only has 5 images. The model is "memorizing" the answer for each image. 

In order to minimize the chance of overfitting, increase the number of images. There should be several thousand example images for each category of object.