How to interpret my neural network with high accuracy but low probability in test results
First possibility:

How well is your dataset balanced? Are there mostly 0s? If I understand your graph well, it would seem that is the case.

If you in fact have 90% 0 and 10% 1 in your dataset, then your results would indicate the model is actually just performing random selection. This means it hasn't really learny anything about the data.

Second possibility:

If your training accuracy is high, but your test accuracy is low, this generally does point to overfitting, as you suspect.

Your model is essentially memorising the training data set, making it able to simply recall the correct result quite reliably, but without learning the underlying function that describes the dataset.

Here are a few things that might be behind this:


training for too many epochs
using a model that too big and complex
not using enough regularisation




I would recommend first checking the class balances of your dataset. If it highly imbalanced, you could look into performing stratified sampling to create your train/test split. This ensures that each batch has roughly the same number of 0 and 1 samples.

Secondly, you might consider using a slightly larger network. In the first instance, this could be as simple as increasing the number of neurons in each of your Dense layers. I would suggest something like (100, 50, 1) to begin with. That will immediately tell you if the size of the model is restricting learning.