How to reduce variance of the model loss during training?
There are a few things you can play with:


Try reducing the learning rate, or increasing decay.
Try using regularization(L1/L2 or dropout)
Try using momentum(your model may be stuck in a local minima)
Adjust other hyperparams(nodes, layers, batch size, etc.)


Unless you have some knowledge about the specific cause of high loss variance, the above steps in some amount should get you where you need to go.