Regarding the choice of cost function in adaptive control - squared error vs absolute error
This is an interesting question since both squared error and absolute error are convex functions, so they are both going to give the optimal solution when minimized. My intuition is that the $\ell_2$-norm (sum of squared values of error) converges to zero more quickly than the $\ell_1$-norm (sum of absolute values of error) when the search direction is right. For example, if the error is $\epsilon = y - Ax$. If $y$ is very close to $Ax$ and $0 &lt; |y-Ax| \leq 1$, then $\epsilon^2$ is much smaller than $|\epsilon|$. Similarly, if $|y-Ax| &gt; 1$, then $\epsilon^2 &gt; |\epsilon|$. So the $\ell_2$ norm penalizes large errors more, and small errors less than the $\ell_1$ norm. This can also be understood by looking at the graphs of the two functions when $\epsilon \in \mathbb{R}$.

Of course the continuity of the derivative is another factor. The derivative of the absolute value technically does not exist at the minimum.