title: When to define multiple Residual blocks in Ceres?
body: <p>I'm going through the Ceres Solver <a href="http://ceres-solver.org/nnls_tutorial.html#powell-s-function" rel="noreferrer">tutorial</a>.</p>
  
  <h3>Powell's Function</h3>
  
  <p>Powell's function maps from <code>R^4 -&gt; R^4</code>, so it seems intuitive to define <em>one</em> residual block that takes in a 4-element array <code>x</code> and fill in a 4-element array <code>residual</code>.</p>
  
  <p>Instead, the example in the tutorial defines 4 different residual blocks that map <code>R^2 -&gt; R^1</code>.</p>
  
  <p>Of course if we are trying to minimize <code>1/2 || F(x) ||^2</code>, then minimizing each element of <code>F</code> will implicitly yield the same solution as minimizing <code>1/2 || F(x) ||^2</code> directly (ie. my suggestion is to return a single residual vector <code>F</code> instead of <code>F1</code>...<code>F4</code> separately). (I have verified this by using the below cost functor).</p>
  
  <pre><code>struct F {
      template&lt;typename T&gt;
      bool operator() (const T* const x, T* residual) const {
  
          residual[0] = x[0] + 10.0 * x[1];
          residual[1] = sqrt(5.0) * (x[2] - x[3]);
          residual[2] = (x[1] - 2.0*x[2]) * (x[1] - 2.0*x[2]);
          residual[3] = T(sqrt(10.0)) * (x[0]  - x[3]) * (x[0] - x[3]);
  
          return true;
      }
  };
  </code></pre>
  
  <ol>
  <li><p>What's the advantage of defining separate residual blocks (and implicitly parameter blocks) for each element the residual vector <code>F</code>?</p></li>
  <li><p>If residual <code>F1</code> depends on parameters <code>x1</code> and <code>x2</code> and residual <code>F2</code> depends on <code>x3</code> and <code>x4</code>, will the cost of <code>F</code> with respect to <code>x1</code> impact the value of <code>x3</code>?</p></li>
  </ol>
  
  <h3>Curve Fitting</h3>
  
  <p>The other example attempts to find parameters <code>m</code> and <code>c</code> to curve <code>y=e^(mx + c)</code>.</p>
  
  <p>It defines some <code>ExponentialResidual</code> which simply outputs <code>T(y_) - exp(m[0] * T(x_) + c[0])</code> where <code>(x_, y_)</code> is a data point.</p>
  
  <p>They then proceed to add one residual block <em>for every observation</em></p>
  
  <pre><code>double m = 0.0;
  double c = 0.0;
  
  Problem problem;
  for (int i = 0; i &lt; kNumObservations; ++i) {
    CostFunction* cost_function =
         new AutoDiffCostFunction&lt;ExponentialResidual, 1, 1, 1&gt;(
             new ExponentialResidual(data[2 * i], data[2 * i + 1]));
    problem.AddResidualBlock(cost_function, NULL, &amp;m, &amp;c);
  }
  </code></pre>
  
  <ol>
  <li>Although I was too lazy to reproduce this example myself, I suspect that this could also be accomplished with just one residual block that maps <code>R^2 -&gt; R^1</code>, where the 1D residual is just the sum of all <code>T(y_) - exp(m[0] * T(x_) + c[0])</code> for all <code>(x_, y_)</code> ? Was it necessary to define a residual block for every observation?</li>
  </ol>
  
  <p>Thanks for reading this long-is post! </p>