title: Multidimensional minimization with the Gnu Scientific Library
body: <p>So I have a problem with the multidimensional minimization procedures in the GSL (the one I'm trying to use is <code>gsl_multimin_fdfminimizer_vector_bfgs2</code>, but the others produce the same problem). I have defined my target function and its gradient and I can start the iteration, but the algorithm always stays at the same point.</p>
  
  <p>Now, I thought I would have a math error in my function or gradient somewhere. However, for a specific set of data I know the optimum and plugging in those values produces a gradient that is 0 everywhere.</p>
  
  <p>If I start the iteration at the optimum, the algorithm correctly stops and reports success, but if I disturb the starting vector only minimally, the iteration runs forever and never finds the optimum.</p>
  
  <p>I also reimplemented the function and gradient in R. I checked that the C version and the R version produce the same output for the same input, which they do. Using the BFGS algorithm in R, the optimum is found in just a few iterations.</p>
  
  <p>I initialize the algorithm with</p>
  
  <pre><code> const gsl_multimin_fdfminimizer_type *T;
   gsl_multimin_fdfminimizer *s;
  
   gsl_multimin_function_fdf lindsey_func;
   const gsl_vector * p[] = {x, y};
  
   lindsey_func.n = J;
   lindsey_func.f = lindsey_loglik;
   lindsey_func.df = lindsey_gradient;
   lindsey_func.fdf = lindsey_gf;
   lindsey_func.params = p;
  
   T = gsl_multimin_fdfminimizer_vector_bfgs2;
   s = gsl_multimin_fdfminimizer_alloc(T, J);     
  
   gsl_multimin_fdfminimizer_set(s, &amp;lindsey_func, beta0, .001, 1e-4);
  </code></pre>
  
  <p>and the iteration looks like this:</p>
  
  <pre><code> do
   {
    iter++;
    status = gsl_multimin_fdfminimizer_iterate(s);
    printf("%d\n", status);
    if (status)
         break;
  
    status = gsl_multimin_test_gradient(s-&gt;gradient, 1e-3);
  
    if (status == GSL_SUCCESS)
    {
       printf("Minimum found at: ");
       printf("[");
       for (j = 0; j &lt; J; j++)
       {
        printf("%.5f, ", gsl_vector_get(s-&gt;x, j));
       }
       printf("]\n");
    }
   } while (status == GSL_CONTINUE &amp;&amp; iter &lt; iter_max);
  </code></pre>
  
  <p>The only parameter that I can tweak are the initial step size and the accuracy of the line minimization in <code>gsl_multimin_fdfminimizer_set (gsl_multimin_fdfminimizer * s, gsl_multimin_function_fdf * fdf, const gsl_vector * x, double step_size, double tol)</code>. Trying different values there has a small effect (sometimes the algorithm moves a little bit), but I can't find any values that actually make the algorithm move around.</p>
  
  <p>Thanks for any suggestions!</p>