Homemade deep learning library: numerical issue with relu activation
I don't know exactly what the problem is, but maybe you could try checking the value of your gradients and see if they change a lot around the 8th epoch.