Understanding general approach to updating optimization function parameters
I reached the same conclusion for a general case below. However, in practice, at least in the case of neural networks, weights are initialized randomly; initializing to the same value is avoided.

A general problematic case

Suppose data is $x=(x_0, x_1)$, where $x_0$ is the vector of features and $x_1$ is the outcome. Let the loss function be:

$$J_{\theta}(x) = (x_1 - f_{\theta}(x_0))^2$$

Only one data point $x$ is considered for simplicity (for a set of points, conclusion would be the same since the argument applies to each term in the summation).

Each parameter $\hat{x}_i$ could be a vector or a single parameter. Gradient of loss would be:

$$\nabla_{\hat{x}_i}J_{\theta}(x) = 2\nabla_{\hat{x}_i}f_{\theta}(x_0)(x_1 - f_{\theta}(x_0))$$

According to this gradient, if two parameters $i$ and $j$ have symmetric roles in $f$, i.e. $$\nabla_{\hat{x}_i}f_{\theta} = \nabla_{\hat{x}_j}f_{\theta},$$ their  corresponding loss gradient will also be the same since the components $\nabla f_{\theta}(x_0)$, $x_1$, and $f_{\theta}(x_0)$ are all the same. A concrete example would be a neural network with equal weights and mean squared loss function. All weights between two specific layers would have the same role in the network, thus they will remain equal after each update. However, in practice, weights of neural networks are initialized randomly which breaks this role symmetry between the weights.

A specific counterexample

For example, consider 2D data $x=(x_0, x_1)$ and two 1D parameters $\theta=[\hat{x}_0, \hat{x}_1]$, and let the loss function be:
$$J_{\theta}(x)= \hat{x}_0 x_0 + \hat{x}_1 x_1,$$
for a batch of one point $x$ (this is for simplicity to avoid a summation over points in the batch).

The gradient w.r.t. parameters is:
$$\frac{\partial J_{\theta}(x)}{\partial \hat{x}_0} = x_0,\mbox{and }\frac{\partial J_{\theta}(x)}{\partial \hat{x}_1} = x_1,$$

This illustrates the dependency of gradient on data $x=(x_0, x_1)$. Now, if both parameters are zero, i.e. $\hat{x}_0=\hat{x}_1=0$, the gradient is still different and non zero. More specifically, suppose learning rate is $\lambda$, the next values for parameters would be:
$$\hat{x}'_0 = \hat{x}_0 - \lambda\frac{\partial J_{\theta}(x)}{\partial \hat{x}_0} = 0 - \lambda x_0 \neq 0 - \lambda x_1 = \hat{x}_1 -\lambda\frac{\partial J_{\theta}(x)}{\partial \hat{x}_1} = \hat{x}'_1$$

But, what if $x_0=x_1$?
In this case, parameters always remain the same if we always use specific data point $x$ to update the parameters. However, this case is pathological (unlikely). Because, to let this equality keep going, any other data point $y$ that we peak must satisfy $y_0=y_1$ too. So in this example, the problem is unlikely to happen.