Gradient for starting Backpropagation I was reading this nice tutorial about Pytorch's basics:

https://pytorch.org/tutorials/beginner/pytorch_with_examples.html

In the first example (pure Numpy), the author starts the backward phase by setting as his "first" gradient the numerical result of the loss function calculation multiplied by the constant 2:

grad_y_pred = 2.0 * (y_pred - y)


Why does he multiply by 2?
This is because of the loss assumption is the (Mean) Squared Error $\mathcal{L} = (\hat{y} - y)^2$ and the derivative is

$$
\frac{\partial}{\partial \hat{y}} \mathcal{L} = 2 (\hat{y} - y)
$$

which is then passed "backward" for use in the chain rule.