title: Linear regression with Lasso penalty needs to increase iterations, Scikit-learn
body: <p>I am using Linear regression with Lasso implemented in Scikit-learn package. </p>
  
  <pre><code>linear_regress = linear_model.Lasso(alpha = 2)
  linear_regress.fit(X, Y)
  </code></pre>
  
  <p>For X, there is 7827 examples and 758 features.
  However I got a warning: </p>
  
  <pre><code>Objective did not converge for target 0, you might want to increase the number of iterations ' to increase the number of iterations')
  </code></pre>
  
  <p>Meanwhile, the MAE of the cross validation is 0.00304247702091</p>
  
  <p>Then, I followed its advise to increase the number of iterations. (I assume that I am doing correctly):</p>
  
  <pre><code>linear_regress = linear_model.Lasso(alpha = 2, max_iter = 100000, tol = 1e-20)
  </code></pre>
  
  <p>But the warning is still there and the MAE increases to 0.0191056040626, which is even worse.</p>
  
  <p>So does anyone know how to fix this problem?</p>
  
  <p>By the way, For the result of cross validation, the MAE of training data is much smaller than that of test data, for instance (alpha=2):</p>
  
  <pre><code>The MAE on the TRAINING data is 6.3462754706e-14
  The MAE on the TEST data is 0.238521024414
  </code></pre>
  
  <p>I assumed that overfitting exists. But increasing alpha does not help much, for instance(alpha=5)</p>
  
  <pre><code>The MAE on the TRAINING data is 1.29613883816e-13
  The MAE on the TEST data is 0.0677816327262
  </code></pre>
  
  <p>Increasing alpha also makes the average MAE increases.</p>
  
  <p>Thanks in advance!</p>