Why doesn't relu cause back propegation to get stuck
Normally, any ReLU unit will return 0 for some samples and not for other samples (in particular, it'll probably return 0 for about half of the samples).  Because of how SGD works, it won't get stuck: the former samples will yield a contribution to the gradient of 0, but the latter samples will have a non-zero gradient, so the weights and bias will be updated for the latter samples.  So it's not a problem; the weights and bias still get updated for many of the samples.
That said, there can be cases where this problem can occur.  If the ReLU unit outputs 0 for all (or most) samples, then SGD won't update the weights, and learning for that neuron will get stuck.  This is sometimes called the &quot;dying ReLU&quot; problem.  See, e.g., https://datascience.stackexchange.com/q/5706/8560 and https://en.wikipedia.org/wiki/Rectifier_(neural_networks)#Potential_problems.   Typically we're able to avoid this problem by appropriate initialization of the weights and by suitable choice of learning rate for SGD, so it's usually not a big deal in practice if you choose those appropriately, but it can happen.