Periodic pattern in loss when training neural networks using pytorch
I am using Pytorch train a model on MNIST, the loss curve has the periodic pattern shown in the figure. I've double checked the data loader and the dataset is shuffled in each epoch. Any suggestions for the possible reason? Thanks. 

Loss curve during training, training loss in blue and test loss in red

What I understand from the figure is that your loss is oscillating. So try decreasing your learning rate and also some momentum term if available. I cannot guarantee you that this will work but hoping that it works it worth giving a try. Most of the things in Deep Learning can be explained, its just trial and error. Next time please ask such questions in https://ai.stackexchange.com or https://datascience.stackexchange.com.
