Accuracy goes straight for about 200 epochs then start increasing Can anyone explain the following observation?
Why did the accuracies keep to be a straight line with a very smooth decrease of loss?
By the way, why is the loss lines so beautifully smooth for the first 400 epochs?
Is this because of the learning rate or other reasons?
The accuracy depends on a threshold, whereas the loss doesn't. ML software tends to assume a threshold of 0.5, which is not a good fit in cases where there's some class imbalance.
I believe that, until epoch 500, your model is learning (loss is going down), but the default threshold doesn't allow you to see it in terms of accuracy.
If you pick another threshold you might see different results.
Regarding the loss going from smooth to noisy, it might be that it is learning the &quot;easy cases&quot; first, and then decreasing very easily and smoothly, and after epoch 400-500 it starts to overfit some of &quot;hard cases&quot;, thus the loss becoming noisier.