title: Neural network, is it worth changing learning rate and momentum over time
body: <p>Is it worth to change learning rate after certain conditions are met? And how and why to do it? For example net will start with high learning rate and after squared error is low enough learning rate will drop for better precision or learning rate should increase to jump-out of local minima?. Wouldn't it cause over-fitting? And what about momentum?  </p>