Policy gradient: why does this converge with Adam and not SGD?
Stochastic gradient descent (SGD) can get stuck in saddle points which results in the model not converging. A saddle points gradient is where zero in many directions but not all directions.
Adam adaptively changes the learning rate based on past gradients. This additional information can allow the model to escape saddle points, thus the model is more likely to converge.