title: How do I set the tolerance in scipy.optimize.minimize()?
body: <p>I am using scipy.optimize.minimize to solve for an efficient portfolio.</p>
  
  <p>With the default settings I am frequently running into "BaseException: Positive directional derivative for linesearch" errors for when using certain inputs.  I noticed if I set the tolerance high enough, the problem becomes less but doesn't go away.  Any advice?</p>
  
  <pre><code>import numpy as np
  import pandas as pd
  import scipy.optimize
  
  
  def fx(TOLERANCE):
      #TOLERANCE = 1.5
  
      def solve_weights(R, C, rf, b_):
          def port_mean_var(W,R,C):
              return sum(R*W), np.dot(np.dot(W, C), W)
          def fitness(W, R, C, rf):
              mean, var = port_mean_var(W, R, C)    # calculate mean/variance of the portfolio
              util = (mean - rf) / np.sqrt(var)        # utility = Sharpe ratio
              return 1/util                        # maximize the utility, minimize its inverse value
          n = len(R)
          W = np.ones([n])/n                        # start optimization with equal weights
          #b_ = [(0.,1.) for i in range(n)]    # weights for boundaries between 0%..100%. No leverage, no shorting
          c_ = ({'type':'eq', 'fun': lambda W: sum(W)-1. })    # Sum of weights must be 100%
          optimized = scipy.optimize.minimize(fitness, W, (R, C, rf), 
                                              method='SLSQP', constraints=c_, 
                                              bounds=b_, tol=TOLERANCE)    
          if not optimized.success: 
              raise BaseException(optimized.message)
          return optimized.x
  
      def mean_var_opt2(ret_df, upper_bounds=None):
          R = (ret_df.mean(0)*252).values
          C = (ret_df.cov()*252).values
          rf = 0.0
          if upper_bounds == None:
              upper_bounds = pd.Series(1.0,index=ret_df.columns)
          b_ = [(0.0,float(num)) for num in upper_bounds]
          wgts = solve_weights(R, C, rf, b_)
          return pd.Series(wgts, index=ret_df.columns)
  
  
      np.random.seed(45)
      rets = []
      for i in range(10000):
          rets.append(pd.DataFrame(np.random.randn(100,4)/100.0))
  
      try:
          for i in range(10000):
              mean_var_opt2(rets[i])
      except BaseException as e:
          print e
      finally: print "Tolerance: %s, iter: %s" % (TOLERANCE,i)
  
  for k in [0.001, 0.01, 0.025, 0.05, 0.1, 0.5, 1.0, 5.0, 50.0, 500.0]:
      fx(k)
  
  
  Positive directional derivative for linesearch
  Tolerance: 0.001, iter: 0
  Positive directional derivative for linesearch
  Tolerance: 0.01, iter: 30
  Positive directional derivative for linesearch
  Tolerance: 0.025, iter: 77
  Inequality constraints incompatible
  Tolerance: 0.05, iter: 212
  Positive directional derivative for linesearch
  Tolerance: 0.1, iter: 444
  Positive directional derivative for linesearch
  Tolerance: 0.5, iter: 444
  Positive directional derivative for linesearch
  Tolerance: 1.0, iter: 1026
  Positive directional derivative for linesearch
  Tolerance: 5.0, iter: 1026
  Positive directional derivative for linesearch
  Tolerance: 50.0, iter: 1026
  Positive directional derivative for linesearch
  Tolerance: 500.0, iter: 1026
  </code></pre>