title: Best model for english alphabet and digits recognition
body: <p>I have a fairly big dataset(over 0.5 mln 50x50 images) consisting of 62 classes in total. Images represent english alphabet and digits(all handwritten). Each class has at least 2000 samples.</p>
  
  <p>I've been training Convolutional Neural Network to recognize these images with Tensorflow.</p>
  
  <p><strong>Problem</strong>: After quickly(about 200 training iterations) getting very close to local optima(loss values like 0.01), my classifier is stuck at around 82% accuracy on test set.</p>
  
  <p><strong>Question</strong>: How can I possibly get more accuracy? Is there something I do wrong with CNN? Also, is trying SVM worth it? I give details according to my CNN model below.</p>
  
  <p>Random dataset entry:
  <a href="https://i.stack.imgur.com/ZqwEM.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/ZqwEM.png" alt="enter image description here"></a>
  <strong>Question 2</strong>: Besides rotating patterns, is my pre-processing methodology good? Shall I stretch patterns or leave it as it is now: with left and right margin regions with redundant white pixels.</p>
  
  <p><strong>Details and hyperparams</strong>:</p>
  
  <ul>
  <li>Tensorflow optimizer: AdamOptimizer</li>
  <li>learning rate alpha: <code>0.001</code></li>
  <li>dropout: <code>1.0</code>(no dropout)</li>
  <li>mini batch size: <code>1500</code></li>
  <li>number of convolutional layers: <code>2</code></li>
  <li>number of pooling layers: <code>2</code></li>
  <li>fully connected layers: <code>1</code></li>
  <li>stride: <code>2</code> pixels</li>
  <li>filter size: <code>5</code> pixels</li>
  <li>test/train set proportion: <code>0.2/0.8</code></li>
  </ul>
  
  <p><strong>NOTE</strong>: Paterrns aren't skeletons, they have their original widths. Images are binary with pattern being 0 value and background 1.</p>
  
  <h2>UPDATE</h2>
  
  <p>Here's my <strong>code</strong> responsible for training and a tiny <strong>subset of images</strong>:
  <br><a href="https://drive.google.com/open?id=0B5kuwbyrKVqnTm1PMGZGMUxUNFU" rel="nofollow noreferrer">https://drive.google.com/open?id=0B5kuwbyrKVqnTm1PMGZGMUxUNFU</a></p>
  
  <p>Due to my slow internet connection I cannot afford to upload enough data. <br>However, you can plot these images to decide if further pre-processing is needed.</p>