title: What is loss exactly?
body: <p>I'm training the im2txt sample of tensorflow and it outputs the loss:</p>
  
  <pre><code>INFO:tensorflow:global step 2174: loss = 3.6930 (15.83 sec/step)
  INFO:tensorflow:global step 2175: loss = 3.6651 (15.52 sec/step)
  INFO:tensorflow:global step 2176: loss = 3.5733 (18.25 sec/step)
  INFO:tensorflow:global step 2177: loss = 3.1979 (18.87 sec/step)
  INFO:tensorflow:global step 2178: loss = 2.9362 (15.99 sec/step)
  INFO:tensorflow:global step 2179: loss = 3.6375 (15.65 sec/step)
  </code></pre>
  
  <p>What is loss? How does it relate to the AI:s probability to perform correctly (is there a formula)? What is ususally an acceptable loss?</p>