title: What does iteration in Machine Learning mean?
body: <p>I do not get the idea of <strong>iterations</strong> in machine learning. Since programming is <strong>deterministic</strong> (as in do the same thing each time it is ran), how can <strong>accuracy improve</strong> by running the <strong>same algorithm</strong> over and over again?</p>
  
  <p>Does this iteration concept have anything to do with <strong>RNN</strong>(Recurrent neural network) - <strong>in a sense that they feedback to the same neutrons more times</strong> - models or <strong>SGD</strong>(Stochastic Gradient Descent) -<strong>where they take same samples of training data for efficiency</strong>- ?</p>
  
  <p>Thanks</p>
  
  <p>Edit: What I mean by Iterations is in this toy app I found on this site:
  <a href="https://blog.kovalevskyi.com/rnn-based-chatbot-for-6-hours-b847d2d92c43" rel="nofollow noreferrer">https://blog.kovalevskyi.com/rnn-based-chatbot-for-6-hours-b847d2d92c43</a></p>
  
  <p>What the author did was that he used RNN to create a chatbot. What I fail to understand is how does increasing the number of iterations increase the accuracy of the prediction since the same algorithm is being ran each time.</p>
  
  <p>But from @Spirit_Dongdong's post, it seems that my understanding of iterations (as in what is done in each iteration might be wrong), hence, I am trying to clarify what is meant by an iteration and what is done in an iteration.</p>