Reason for having both low loss and same predicted class?
From your description, it might be an issue of too high learning rate. When this happens, the weights cannot reach closer to the minimum and the loss does not go down.

Accordingly, I would suggest reducing the learning rate to 0.001 and using Adam.