title: Which optimization algorithm is the best for ConvNets on small datasets to avoid overfitting?
body: <p>I am using a ConvNet to classify objects cropped from a sequence of grayscale IR images (binary classification). The network input has size (WxHxN), where N > 3.</p>
  
  <p>The <strong>dataset is small</strong>, only 1-1.5K samples for training. Therefore, I am using a relatively small ConvNet (32,32, 64,64, 128,128, 256,256 + FC 512), with dropout(0.5) in the last 3 layers; implemented with tflearn and/or keras with <strong>TensorFlow</strong>. I also use data augmentation (random blur, horizontal flip, random rotation).</p>
  
  <p>When I use the <strong>Adam optimizer</strong> with learning_rate=<strong>1e-4</strong>, it converges in a few epochs to 99% training accuracy and very small loss, but validation loss keeps increasing and accuracy fluctuates wildly. Eventually, the test set performance is around 50-60 percent, which is almost the same as random guess (binary classification).</p>
  
  <p>I tried L2 weight regularization, batch normalization; different batch sizes, smaller networks, etc. However, it did not work.</p>
  
  <p>Now, I am exploring various optimization algorithms. It seems, with <code>tflearn.optimizers.Nesterov(learning_rate=2e-3, momentum=0.9, lr_decay=0.90, decay_step=100)</code>, the learning is much slower, but smoother; the validation error does not increase as before, the accuracy does not change wildly (mostly, increases slowly or stays the same). The accuracy is better, ~60% at 50 epochs, ~70% at 100 epochs.</p>
  
  <p>Now the question: it seems, the choice of optimization algorithm (and its parameters) affects overfitting. However, I have not seen any work on this. Is there any work/paper/blog that elaborates on this issue? Do you have any experience to share? Which optimization algorithms are good on small datasets, and better at handling overfitting?</p>