title: Gradient Descent vs Stochastic Gradient Descent algorithms
body: <p>I tried to train a FeedForward Neural Network on the MNIST Handwritten Digits dataset (includes 60K training samples).</p>
  
  <p>I each time iterated over <strong>all the training samples</strong>, performing <strong>Backpropagation</strong> for each such sample on every epoch. The runtime is of course too long.</p>
  
  <ul>
  <li>Is the algorithm I ran named <strong>Gradient Descent</strong>?</li>
  </ul>
  
  <p>I read that for large datasets, using <strong>Stochastic Gradient Descent</strong> can improve the runtime dramatically.</p>
  
  <ul>
  <li>What should I do in order to use <strong>Stochastic Gradient Descent</strong>? Should I just pick the training samples randomly, performing <strong>Backpropagation</strong> on each randomly picked sample, instead of the epochs I currently use?</li>
  </ul>