Is the Jacobian matrix calculation process correctly on this energy function? When I implemented the paper Dynamic Fusion published in 2015 CVPR, I came into the trouble with how to solve the energy function:
$$E(W_t, V, D_t, \mathcal{E}) = \text{Data}(W_t, V, D_t) + λ\text{Reg}(W_t, \mathcal{E})$$
Where:

Wt is a set of 4x4 transformation matrix at time t;
V is a set of 3D vertices from canonical model;
Dt is a depth map at time t;
ε is used to find Wt matrices;
Data is a point-to-plane error;
Reg is a smooth term which enforce the two adjacent transformation matrix W to be similar.

To simplify the question, I try to solve the Data term and throw away Reg term.
$$E(W_t, V, D_t, \mathcal{E}) = \text{Data}(W_t, V, D_t)$$
Here is the Data term:
\begin{align*}
r &amp;= \text{Data}(W_t, V, D_t)= \sum_u n_u^{\intercal}\cdot (v_u-vl) \\
where,\text{v}_u &amp;= W_t×\text{v}(u)  \\
\text{n}_u &amp;= W_t×\text{n}(u)  
\end{align*}
All the information but $W_t$ is known, use Gauss-Newton to solve the parameter:
$$\begin{gather*}
\underset{W_t}{\text{minimize}}\ r^2 \\
W_t^{(t+1)} = W_t^t - (J^\intercal J)^{-1}Jr
\end{gather*}$$
It seems to me that I need to calculate the Jacobian matrix $J$, so I expand r in matrix form:
$where\ W_t=$
\begin{bmatrix}
    1  &amp; -\gamma &amp; \beta &amp; t_x \\
    \gamma &amp; 1 &amp; -\alpha &amp; t_y \\
    -\beta &amp; \alpha &amp; 1 &amp; t_z \\
    0 &amp; 0 &amp; 0 &amp; 1
\end{bmatrix}
$v, n, vl$ are all 4x1 vector. (sorry, I don't know how to display matrix multiplication)
If I print all of the calculation, there are about 50 values containing $(\alpha,\beta,\gamma,t_x,t_y,t_z)$ to sum up for one Data term. My question is, can I directly do partial differential to the parameters $(\alpha,\beta,\gamma,t_x,t_y,t_z)$ of a $W_t$ matrix to get Jacobian matrix $J$?  It it correct to update $W_t$ by $W_t^{(t+1)} = W_t^t - (J^\intercal J)^{-1}Jr$?
I cannot believe this is a good approach because it is kind of messy, and maybe inefficient.
Please give me some advice how to solve this energy function, and I am curious why researchers only give energy function without explicit deviation on how to solve the function. Because energy function deviation is a common knowledge? If so, which course teach about it? I really want to take a course. Thanks.
That is not an equation to be solved; it is an objective function to be minimized.  They apparently describe how they optimize it in Section 3.3.3 of the paper.  I suggest trying to understand that section.