How can a neural network get unstuck from a local minimum?
In various ways suchs as

momentum: think of momentum as a means of dampening oscillations and speeding up the iterations, leading to faster convergence. This means replacing gradients with a leaky average over past gradients.
sparse features and preconditioning (Adagrad): decrease the learning rate dynamically on a per-coordinate basis. This means, use the magnitude of the gradient as a means of adjusting how quickly progress is achieved - coordinates with large gradients are compensated with a smaller learning rate.
RMSProp: a combination of momentul and adagrad, combining leaky averages and coefficient-wise preconditioner
Adadelta: The learning rate cannot be parameterised but rather adapts itself given rate of change in the model parameters
Adam: a great algorithm that summarises features from all of the above
Scheduling: decreasing the learning rate during training

sources: https://distill.pub/2017/momentum/, https://d2l.ai/chapter_optimization