title: Python Sklearn Neural Network Classifier Iteration and Loss
body: <p>I was trying to implement a paper I read. Basically it uses 3 neural network classifiers with different parameter to work on the same loan default data with 9 different training-to-testing ratios. </p>
  
  <p>In order to find best parameter, we use following criterion, when(1) <code>max_iteration=25000</code> 
  and (2) Loss value is less than 0.008, we measure the accuracy value, and pick the best.</p>
  
  <p>However, when I try to use python <code>sklearnsklearn.neural_network.MLPClassifier</code> to finish this, I met a problem. When the training-to-test ratio increases, the iterating time the program runs drops dramatically. In the mean while, the loss value increases,</p>
  
  <p><strong>Classifier Performance Table</strong>. </p>
  
  <p><a href="https://i.stack.imgur.com/DENan.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/DENan.png" alt="enter image description here"></a></p>
  
  <p>This is clearly not what I want, the iterating number should keep rising to 25000 before stop.</p>
  
  <p>This is how I defined classifer: </p>
  
  <pre><code>clf1= MLPClassifier(activation='relu',solver='sgd',early_stopping=False,alpha=1e-5,max_iter=25000,\
                  hidden_layer_sizes=(18),momentum=0.7,learning_rate_init=0.0081,tol=0,random_state=3)
  clf2= MLPClassifier(activation='relu',early_stopping=False,solver='sgd',alpha=1e-5,max_iter=25000,\
                  hidden_layer_sizes=(23),momentum=0.69,learning_rate_init=0.0095,tol=0,random_state=3)
  clf3= MLPClassifier(activation='relu',early_stopping=False,solver='sgd',alpha=1e-5,max_iter=25000,\
                  hidden_layer_sizes=(27),momentum=0.79,learning_rate_init=0.0075,tol=0,random_state=3)
  </code></pre>
  
  <p>As you can see, I already set <code>Tolerance=0</code>, so that every time when we iterate, it could surely decrease the loss. And I had tried other value, but still, the iterate number is way smaller than I expected.</p>
  
  <p>Hope someone can help me, thanks!</p>