Why could an overfitted CNN model have a higher validation accuracy?
I will try to answer your question "Why can the overfitted model have a higher validation accuracy?" with an example, then try to answer the remaining questions (TLDR at bottom):

Say we have 2 classes with a 1:1 ratio in both the training and testing sets (each 4 instances).

Perhaps our training/validation curves are just like yours (I highlighted some points for reference).


Going from point A to point B ($\overline{AB}$), we do not observe overfitting:


validation accuracy increases
validation loss decreases


$\therefore$ We might conclude an inverse relationship between validation accuracy/loss.

However, $\overline{BC}$ shows overfitting:


validation accuracy is nearly constant
validation loss increases


This appears to contradict the relationship we observed with the $\overline{AB}$ example!

The reason behind this phenomenon is simple: we cannot assume a linear dependence between the accuracy and loss functions.



Let us define our accuracy and loss functions respectively:

$accuracy:\quad 1-\frac{1}{N}\sum^N_{i=1}|[X_i]-\hat{X}_i|$

$loss \space (MSE):\quad \frac{1}{N}\sum^N_{i=1}(X_i-\hat{X}_i)^2$

$\{X \space | \space 0&lt;X&lt;1\},\qquad\{\hat{X} \space | \space \{0,1\}\}$

We can see that these are linearly independent (which you could verify by calculating their Wronskian). How might this affect the relationship between training/loss curves?

Let us plug-in some numbers to see. Columns {A,B,C} represent the model output vectors for these three epochs and the corresponding ground truth is in $\hat{X}$ column.

\begin{array}{|c|c|c|c|}
\hline
A   &amp; B   &amp; C &amp; \hat{X}\\ \hline
0.6 &amp; 0.3 &amp; 0.3 &amp; 0\\ \hline
0.2 &amp; 0.2 &amp; 0.2 &amp; 0 \\ \hline
0.4 &amp; 0.4 &amp; 0.1 &amp; 1 \\ \hline
0.8 &amp; 0.8 &amp; 0.8 &amp; 1 \\ \hline
\end{array}

\begin{array}{|c|c|c|}
\hline
epoch &amp; accuracy \space calculation &amp; accuracy &amp; loss \space calculation &amp; loss \\ \hline
A &amp; 1-\frac{1}{4} * (1 + 0 + 1 + 0) &amp; 0.50 &amp; \frac{1}{4} * (0.6^2 + 0.2^2 + 0.6^2 + 0.2^2) &amp; 0.2000\\ \hline
B &amp; 1-\frac{1}{4} * (0 + 0 + 1 + 0) &amp; 0.75 &amp; \frac{1}{4} * (0.3^2 + 0.2^2 + 0.6^2 + 0.2^2) &amp; 0.1325 \\ \hline
C &amp; 1-\frac{1}{4} * (0 + 0 + 1 + 0) &amp; 0.75 &amp; \frac{1}{4} * (0.3^2 + 0.2^2 + 0.9^2 + 0.2^2) &amp; 0.2450 \\ \hline
\end{array}

Therefore, $C_{loss} &gt; A_{loss},\quad$even though$\quad C_{accuracy} &gt; A_{accuracy}$!

Of course this is a very simple example, and your situation will have many more complexities, but establishing the linear independence of the validation accuracy and validation loss and understanding these curves is a critical aspect of determining which epoch model to select.



TLDR: With this in mind, you should use the model instance at epoch B. This is the point at which overfitting starts, not at epoch 1250. (although I suggest saving many intermediate models and empirically determining their viability on another dataset).

After epoch B, your validation accuracy is not changing while your training loss decreases. Therefore, your weights are fitting more to the training data without improving your results, ergo overfitting. At epoch 1250, your model is still improving validation accuracy quite dramatically, so stopping training then would yield a very poor model.

As we can see from your validation accuracy curve, this model will probably perform similarly to models beyond epoch B, but there are a couple reasons you might prefer model B.

If you ever wanted to resume training (perhaps you add more data to your training set but do not want to restart completely), then having weights that are not overly fitted (or even vanished!) to the initial dataset would be ideal.

A very similar reason is that, even though this particular dataset shows virtually no difference between the epoch B model and models beyond epoch B, other datasets might. It would then be easier to adjust model B for these new datasets rather than an overfitted model.