Image features (produced by VGG19) do not properly train an ANN in Keras
After doing a bunch of searching and trial &amp; error, I found out that using the sigmoid activation function for all the layers and using a binary_crossentropy loss fixed the issue. The newly added hidden layers were too few and too small for the ReLU function to work.