Title: Faster Convergence & Generalization in DNNs,
Abstract: Deep neural networks have gained tremendous popularity in last few years.
They have been applied for the task of classification in almost every domain.
Despite the success, deep networks can be incredibly slow to train for even
moderate sized models on sufficiently large datasets. Additionally, these
networks require large amounts of data to be able to generalize. The importance
of speeding up convergence, and generalization in deep networks can not be
overstated. In this work, we develop an optimization algorithm based on
generalized-optimal updates derived from minibatches that lead to faster
convergence. Towards the end, we demonstrate on two benchmark datasets that the
proposed method achieves two orders of magnitude speed up over traditional
back-propagation, and is more robust to noise/over-fitting.