Do zero weights receive zero gradient in ReLU neural networks?
The derivative (of a loss function) with respect to $w_i$ can be any value, independent of the value of $w_i$.  (E.g. $f(x)=x$ has derivative 1 everywhere, even at $x=0$.)