Validation loss is neither increasing or decreasing
Usually when a model overfits, validation loss goes up and training loss goes down from the point of overfitting. But for my case, training loss still goes down but validation loss stays at same level. Hence validation accuracy also stays at same level but training accuracy goes up. I am trying to reconstruct a 2D image from a 3D volume using UNet. Same is the behavior when I am trying to reconstruct 3D volume from 2D image but at higher loss and lower accuracy. Can someone explain the curve that why validation loss is not going down from the point of overfitting?


The trends show that your model is overfitting. Ways to overcome overfitting include:

Use data augmentation
Use more data
Use Dropout
Use regularization
Try slowing down your learning rate!

