Which function $(\hat{y} - y)^2$ or $(y - \hat{y})^2$ should I use to compute the gradient?
The derivative of $\mathcal{L_1}(y, x) = (\hat{y} - y)^2 = (f(x) - y)^2$ with respect to $\hat{y}$, where $f$ is the model and $\hat{y} = f(x)$ is the output of the model, is 

\begin{align}
\frac{d}{d \hat{y}} \mathcal{L_1}
&amp;= \frac{d}{d \hat{y}} (\hat{y} - y)^2 \\
&amp;= 2(\hat{y} - y) \frac{d}{d \hat{y}} (\hat{y} - y) \\
&amp;= 2(\hat{y} - y) (1) \\
&amp;= 2(\hat{y} - y) 
\end{align}

The derivative of $\mathcal{L_2}(y, x) = (y - \hat{y})^2 = (y - f(x))^2$ w.r.t $\hat{y}$ is

\begin{align}
\frac{d}{d \hat{y}} \mathcal{L_2}
&amp;= \frac{d}{d \hat{y}} (y - \hat{y})^2 \\
&amp;= 2(y -\hat{y}) \frac{d}{d \hat{y}} (y -\hat{y}) \\
&amp;= 2(y - \hat{y})(-1)\\
&amp;= -2(y - \hat{y})\\
&amp;= 2(\hat{y} - y)
\end{align}

So, the derivatives of $\mathcal{L_1}$ and $\mathcal{L_2}$ are the same.