title: Early stopping : neural networks
body: <p>I'm working on relation classification with the SemEval2010 Task 8 dataset. The dataset is already split into 8'000 samples for the training and 2'717 for the testing. In order to be as fair as possible, I use only my model at the end to computing its performance (F1-Score).</p>
  
  <p>In order to tune my convolutional neural networks, I keep 6'400 samples for the training and 1'600 for the validation. I train the model and after each epoch (~10' of computation) I compute the F1-Score of my predictions.</p>
  
  <p>I read the paper <a href="http://page.mi.fu-berlin.de/prechelt/Biblio/stop_tricks1997.pdf" rel="nofollow noreferrer">http://page.mi.fu-berlin.de/prechelt/Biblio/stop_tricks1997.pdf</a> and stop training when the last 3 performances were increasing (similar to UP in the paper). In the paper, they return the model corresponding to the best performance seen so far.</p>
  
  <p>My question is : in order to be as accurate as possible, we need the whole 8'000 samples for the training. Is it correct to say we will train until the epoch which had the best performance on the validation set and then do the predictions ? Or should we save the model corresponding to the best performance and "waste" 1'600 samples ?</p>