What is the best XOR neural network configuration out there in terms of low error?
The initialization of the weights has a big impact on the results. I'm not sure specifically for the XOR gate, but the error can have a local minimum that the network can get &quot;stuck&quot; in during training. Using stochastic gradient descent can help give some randomness that gets the error out of these pits. Also, for the sigmoid function, weights should be initialized so that the input to the activation is close to the part with the highest derivative so that training is better.