title: Keras cifar10 example validation and test loss lower than training loss
body: <p>I'm playing around with the cifar10 example from Keras which you can find <a href="https://github.com/fchollet/keras/blob/master/examples/cifar10_cnn.py" rel="noreferrer">here</a>. I've recreated the model (i.e., not same file but everything else pretty much the same) and you can find it <a href="https://github.com/sudarshan85/dl-projects/blob/kcifar/keras-cifar10/cifar10_cnn.py" rel="noreferrer">here</a>.</p>
  
  <p>The model is identical and I train the model for 30 epochs with 0.2 validation split on the 50,000 image training set. I'm not able to understand the result I get. My validation and testing loss is lesser than the training less (inversely, training accuracy is the lower compared to the validation and testing accuracy):</p>
  
  <blockquote>
  <pre><code>                      Loss       Accuracy
     Training          1.345          0.572
   Validation          1.184          0.596
         Test           1.19          0.596
  </code></pre>
  </blockquote>
  
  <p><a href="https://i.stack.imgur.com/hPVZZ.png" rel="noreferrer"><img src="https://i.stack.imgur.com/hPVZZ.png" alt="Model Training and Validation loss and accuracy"></a></p>
  
  <p>Looking at the plot, I'm not sure why the training error starts increasing again so badly. Do I need to reduce the number of epochs I train for or maybe implement early stopping? Would a different model architecture help? If so, what would be good suggestions?</p>
  
  <p>Thanks.</p>