Question of using gradient descent instead of calculus. I checked previous questions there are still points to clarify
If I understand correctly, your question is Why do we solve gradient descent problems iteratively rather than setting the derivative to 0 and solving for the weights which minimize the cost function?

We can't do this (in all but extremely simple problems such as linear regression) because there is no easy closed-form solution when you set the derivative equal to 0.

Let's take a look at a simple example: let's say you have a 1-layer neural net with no bias and a sigmoid activation function.  Your network output is $\hat{y} = \sigma(WX)$.  If you use L2 loss, your cost function would be $L = (\sigma(WX) - y)^2$.  (For simplicity we'll assume that we're only using a batch size of 1).

Let's take the derivative with respect to our weights $W$ and use the chain rule
:
$\begin{eqnarray*}
\frac{dL}{dW} &amp;=&amp; 2(\sigma(WX) - y)\frac{d}{dW}\sigma(WX)\\
&amp;=&amp; 2(\sigma(WX) - y)\sigma'(WX)\frac{d}{dW}(WX)\\
&amp;=&amp; 2(\sigma(WX) - y)\sigma(WX)(1-\sigma(WX))\frac{d}{dW}(WX)\\
&amp;=&amp; 2(\sigma(WX) - y)\sigma(WX)(1-\sigma(WX))X\\
\end{eqnarray*}$

At this point, we get a bit stuck.  If we set this derivative equal to 0, it still won't be easy to solve for $W$ since $W$ appears 3 times in this equation and can't easily be isolated by itself on one side of the equation.

Since we cannot solve for the cost-minimizing weights directly, instead we use gradient descent.  We take the derivative of the cost function to tell us which direction we should change each weight in order to most decrease the cost.  The actual size of the update is determined by the learning rate $\alpha$.