Why do neural networks trained on identical datasets and with identical hyper-parameters have different performances? I found that fully connected neural networks trained on identical data sets with identical hyper-parameters can have different performances or accuracies (7-8% of deviation). Is this an unusual phenomenon? What is the cause? What is an acceptable deviation?
This should simply be due to the fact that network weight initialization is random.