Cost function for adaptive algorithms
In a very simple form: Gaussian-like noises are frequent, and derivatives of squared functions are easy to minimize.

In very simple words: error, directly, cannot be used as a criterion. You can minimize a signed function to $-\infty$, which is not really useful here. As least, you need a cost function with a lower bound.

Power (if by that you mean the squared error) is an increasing function of the error, and positive. So if you minimize power, you can expect to minimize, somehow, a measure of the error (more precisely its absolute value). 

The first problem is that the absolute value is not differentiable around zero, and can be difficult to optimize (one may look for a zero in the derivative). And when can you very easily find a zero in a derivative? When it is linear, because since Laplace, Gauss, Seidel such systems can be solved efficiently (matrix inversion). What are the functions that yield linear derivatives? Squares, parabolas...

Basically, for a while, squared functions were the only ones one could solve easily. The standard mean is just the minimum of the sum of differences squared. Interestingly, in linear cases, unrelated disturbances can add with a Gaussian distributions. And Gaussian distributions are well-related to squared errors. 

But you are interested, recent advances in optimization allow to optimize more easily other types of functions, including absolute value loss criteria   or penalties.