Accuracy noise patterns during model training I'm training a logistic regression model on a small dataset. I have about 1300 samples that I split into a training and a testing set (70% and 30% respectively).

The training seems ok, however when I plot the accuracy of my model w.r.t. the epoch, some repeating noisy patterns appear at the end well after the accuracy is stabilised (after 800 epochs, see images below).

The training is done with an Adam optimizer. I'm using a learning rate of 0.04 and a weight decay of 0.07 that I found after doing a random search.

Is it something that may happen when training and is without consequence or does it reflect an issue with my data / training / software implementation ?
It is likely that in your training phase you are reaching local minima, then since you 'persist' at each epoch you get off of your point and then after the 800 epochs you reach it again.

Look at the image below, imagine you reach the blue point, but you keep persisting, looking for another point, then you will be off of your track but it will eventually come back. The pattern will be more clear in logistic regression since theoretically you are applying gradient descent to a simpler function than a complex neural network.