title: How to change max_iter in optimize function used by sklearn gaussian process regression?
body: <p>I am using sklearn's GPR library, but occasionally run into this annoying warning:</p>
  
  <pre><code>ConvergenceWarning: lbfgs failed to converge (status=2):
  ABNORMAL_TERMINATION_IN_LNSRCH.
  
  Increase the number of iterations (max_iter) or scale the data as shown in:
      https://scikit-learn.org/stable/modules/preprocessing.html
    _check_optimize_result("lbfgs", opt_res)
  </code></pre>
  
  <p>Not only can I find almost no documentation on this warning, max_iter is not a parameter in sklearn's GPR model at all. 
  I tried to rescale the data as suggested, but it didn't work and frankly I didn't understand it (do I also need to scale the output? Again, little documentation).</p>
  
  <p>Increasing the maximum iterations in the optimization process makes sense, but sklearn does not appear to have a way to do that, which is frustrating because they suggest it in response to this warning. </p>
  
  <p>Looking at the GPR <a href="https://scikit-optimize.github.io/stable/_modules/sklearn/gaussian_process/_gpr.html" rel="noreferrer">source code</a>, this is how sklearn calls the optimizer,</p>
  
  <pre><code> def _constrained_optimization(self, obj_func, initial_theta, bounds):
          if self.optimizer == "fmin_l_bfgs_b":
              opt_res = scipy.optimize.minimize(
                  obj_func, initial_theta, method="L-BFGS-B", jac=True,
                  bounds=bounds)
              _check_optimize_result("lbfgs", opt_res)
              theta_opt, func_min = opt_res.x, opt_res.fun
          elif callable(self.optimizer):
              theta_opt, func_min = \
                  self.optimizer(obj_func, initial_theta, bounds=bounds)
          else:
              raise ValueError("Unknown optimizer %s." % self.optimizer)
  
          return theta_opt, func_min
  </code></pre>
  
  <p>where <code>scipy.optimize.minimize()</code> has default values of </p>
  
  <pre><code>scipy.optimize.minimize(fun, x0, args=(), method='L-BFGS-B', jac=None, bounds=None, 
  tol=None, callback=None, options={'disp': None, 'maxcor': 10, 'ftol': 2.220446049250313e-09,
  'gtol': 1e-05, 'eps': 1e-08, 'maxfun': 15000, 'maxiter': 15000, 'iprint': -1, 'maxls': 20})
  </code></pre>
  
  <p>according to the scipy <a href="https://docs.scipy.org/doc/scipy/reference/optimize.minimize-lbfgsb.html" rel="noreferrer">docs</a>.</p>
  
  <p>I would like to use exactly the optimizer as shown above in the GPR source code, <strong>but change maxiter to a higher number.</strong> In other words, I don't want to change the behavior of the optimizer other than the changes made by increasing the maximum iterations. </p>
  
  <p>The challenge there is that other parameters like <code>obj_func, initial_theta, bounds</code> are set within the GPR source code and are not accessible from the GPR object.</p>
  
  <p>This is how I'm calling GPR, note that these are mostly default parameters with the exception of n_restarts_optimizer and the kernel. </p>
  
  <pre><code>for kernel in kernels:
      gp = gaussian_process.GaussianProcessRegressor(
                      kernel              = kernel,
                      alpha               = 1e-10,
                      copy_X_train        = True,
                      optimizer           = "fmin_l_bfgs_b",
                      n_restarts_optimizer= 25,
                      normalize_y         = False,
                      random_state        = None)
  </code></pre>