   Stuck in a local minima with backprop
  Hi there,I have my NN being trained using backprop and the data set is quite large. I noticed that when I have large data, it tends to get stuck forever....in other words the sum squared error never converges to the desired threshold...it just fluctuates up and down non stop. My NN works fine with small data but not with large.Ive tried using momentum on it to see if will work but it does not. Ive tried early stopping and all that, that did was result in the NN not really learning anything.Can anybody help?ThanksDarkStarUK   -------------------------------Loves cross-posting because it works 
  What do you mean by desired threshold?  Do you know those particular NN can solve this problem?  Have you tried using lots of different random weights to start, to get an idea of the possibilities?  How about changing the parameters of the NN too?  Have you tried RProp in batch training, that usually finds a near-minimal error.Alex ;  for quite large training sets, you have to set the learning rate really low, try to adjust there a bit. then, if you want to stay with bprop ( just almost the simplest math  ), you could try bprop with momentum, that mostly is able to handle local minima.another possibility might be, that your net is just too little to learn all those information ;  As Alex suggested, try changing your initial conditions for the untrained network. If that fails, consider adding an extra node in the hidden layer. It may be that the resolution of your network is too low to uniquely identify the decision boundaries in your large data set (particularly if there is a lot of noise in the input set).Cheers,Timkin 