How parameter adjustment works in Gradient Descent?
Imagine we have the curve $f(x) = x^2$, and we want to find the minimum of this function. The derivate of $f$ with respect to $x$ is $2x$. Now, gradient descent works by updating our current estimate of the minimum, say $c_t$, by the following iterative process
$$c_{t+1} = c_t - \alpha \times \nabla_xf(x=c_t),$$
where $\alpha$ is some constant to control how much we shift towards the gradient.
Intuitively, this should make sense. Imagine our current estimate of the minimum is $c_t = -1$. The update would then give us $c_{t+1} = -1 - \alpha \times -2 = -1 + 2\alpha &gt; -1$. As you can see, the update has shifted our estimate in the direction of the minimum. If our estimate were $+1$ then you can probably see that the update would again have shifted us in the direction of the minimum again.
Now, what happens in machine learning is we have a loss function $L$ that we typically want to find the minimum of in terms of the parameters of our model. By applying gradient descent on the loss function as we did above with $f(x)$ we iteratively apply the update rule which will eventually lead us to the minimum of our loss function with respect to the weights. The process is exactly the same as above except it is likely to happen in higher dimensions, where the derivative of $f$ becomes a vector of partial derivates. Note that whilst $w_i$'s are scalar values, we are not differentiating these values, rather we are differentiating the loss function with respect to these scalars.
I would recommend you to try this out with a simple linear model of maybe 2 or 3 parameters. After a quick Google I found this article that may be useful.