What's the correct reasoning behind solving the vanishing/exploding gradient problem in deep neural networks.?
I think the two aspects you mention are two faces of the same medal: if your weights are too high or low, the activation of a layer ends up being too high or low. If you use $tanh(z)$ or $sigmoid(z)$ as activation functions, you'll end up with values that will have a derivative almost to zero, and the gradient will vanish as it is back-propagated into the network.
By using $relu(z)$, instead, you use a linear function for $z&gt;0$ and therefore the derivative is constant as the activation grows. This helps the gradient which does not approach zero while it is back-propagated.

This works in every case: both if you use weights that are too high or too small.
Tanh and sigmoid are similar in the sense that they are basically linear near $z=0$, but almost flat when $|z|&gt;&gt;0$, ReLU solves the problem in the latter case, because it becomes linear for any $z&gt;0$.

On the other hand, another strategy is to prevent $z$ from being too large or too small, and therefore you attempt to keep it near the origin ($z=0$). In this way, both tanh and sigmoid are linear, as ReLU would have been for any $z&gt;0$.

The expected result is the same: the derivative of the function is almost linear, and therefore the gradients can traverse the network without vanishing or exploding.

Of course, there are other factors here that must be considered (e.g. ReLU clamps to 0 every negative value, which has other sort of effects), but beside these, in both cases you get a similar outcome (non-vanishing gradients) with different approaches (by tweaking the activation function or by tweaking the weights).