Regularization term moving the hyperbolic plane When we train a network, we usually have an error function which usually looks like this:

$$Error = (output - target)^2$$

When we add a regularization term, it looks something like this:

$$Error = (output - target)^2 + reg term$$

I understand that there are many different ways of defining the reg term, but in this example, we can make it the sum of all weights.

When we do this, are we just not moving the hyperbolic plane up and down? The slope of it should stay the same everywhere and thus not really make a difference in finding a minimum of the error function.
The point of the regularization term is usually to keep the network from overfitting by keeping the weights vector with low absolute values or sparse. Therefore I would say a sum of all weights is not a good regularization term, because even extreme positive and negative weights can sum to zero. In the regularization term, the $L_2$, $L_1$, $L_0$ norms are commonly used (read about it here). 

In a side note, if I understood you correctly, I would suggest to avoid imagining the target function (or loss function) as a hyperbolic plane, since usually DNNs contain non linear activation functions like ReLU.