title: Gradient Descent vs Adagrad vs Momentum in TensorFlow
body: <p>I'm studying <em>TensorFlow</em> and how to use it, even if I'm not an expert of neural networks and deep learning (just the basics).</p>
  
  <p>Following tutorials, I don't understand the real and practical differences between the three optimizers for loss. I look at the <a href="https://www.tensorflow.org/versions/r0.7/api_docs/python/train.html#optimizers" rel="noreferrer">API</a> and I understand the principles, but my questions are:</p>
  
  <p><strong>1. When is it preferable to use one instead of the others ?</strong></p>
  
  <p><strong>2. Are there important differences to know ?</strong></p>